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Why  GAO  Did  This  Study 

In  the  1 1  years  since  September  1 1 , 
2001,  the  U.S.  government  has  put 
forth  extensive  efforts  to  protect  the 
nation’s  aviation  sector  and  airspace. 
These  efforts  include  air  sovereignty 
alert  (ASA)  operations,  for  which  the 
Air  Force  provides  personnel  and  fully 
fueled,  fully  armed  aircraft  sitting  on 
constant  alert  at  18  sites  across  the 
United  States.  In  2009,  GAO  found 
shortcomings  in  the  Department  of 
Defense’s  (DOD)  management  of  ASA 
operations,  leading  to  a  number  of 
GAO  recommendations.  For  this 
report,  GAO  examined  the  extent  to 
which  (1)  the  Air  Force  has 
implemented  GAO’s  2009 
recommendations,  (2)  DOD  has 
implemented  a  risk-based 
management  approach  for  ASA 
operations,  and  (3)  the  Air  Force  has 
accurately  identified  expenditures  for 
ASA  operations.  To  do  so,  GAO 
analyzed  relevant  strategies,  planning 
documents,  guidance,  and  expenditure 
data;  and  interviewed  North  American 
Aerospace  Defense  Command 
(NORAD),  Air  Force,  National  Guard 
Bureau,  and  other  DOD  officials. 

What  GAO  Recommends 

Congress  may  wish  to  consider 
requiring  the  Air  Force  to  fully 
implement  GAO’s  2009 
recommendations.  In  addition,  GAO 
recommends  that  DOD  improve  its  risk 
management  of  ASA  operations  and 
improve  the  Air  Force’s  ability  to 
accurately  identify  ASA  expenditures. 
DOD  fully  or  partially  agreed  with  all  of 
GAO’s  recommendations. 
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What  GAO  Found 

The  Air  Force  has  not  fully  implemented  the  recommendations  from  GAO’s  2009 
report.  With  regard  to  GAO’s  recommendation  that  the  military  services  should 
formally  assign  ASA  duties  to  the  units  that  consistently  conduct  them  and 
ensure  that  the  readiness  of  those  units  is  fully  assessed,  the  Air  Force  did  so. 
However,  the  National  Guard  Bureau  is  considering  reversing  that  action 
because  it  believes  that  the  recommendation  can  be  better  addressed  through 
the  Air  Force’s  standard  deployment  process.  The  Air  Force  has  also  not 
established  a  timetable  to  implement  ASA  as  a  steady-state  mission;  has  not 
developed  and  implemented  a  plan  to  recapitalize  the  aging  fighter  aircraft  that 
conduct  ASA  operations  before  the  end  of  their  service  lives;  and,  when  ASA 
units  are  deployed  to  support  other  ongoing  operations,  the  Air  Force  continues 
to  identify  replacement  units  to  perform  the  ASA  mission  on  an  ad  hoc  basis.  All 
of  the  above  were  related  to  recommendations  GAO  made  to  the  Air  Force  in  its 

2009  report.  Separately,  GAO  found  considerable  confusion  about  the 
capabilities  associated  with  ASA  operations  in  part  because,  in  September  201 1 , 
NORAD  stopped  using  the  term  “air  sovereignty  alert”  and  created  a  new  term, 
“aerospace  control  alert”  (ACA),  without  clearly  defining  ACA  or  the  missions  that 
are  now  included  within  it. 

DOD  has  taken  a  series  of  actions  for  ASA  operations  that  are  consistent  with  a 
risk-based  management  approach.  However,  several  key  actions  have  yet  to  be 
taken  that  would  enable  the  department  to  better  balance  risk  and  costs.  Risk- 
based  management  includes  conducting  routine  risk  assessments  that  evaluate 
threats,  vulnerabilities,  and  criticality  of  assets,  as  recommended  in  GAO’s  2009 
report,  and  selecting  between  alternative  courses  of  action  to  mitigate  risk  and 
make  decisions  about  allocating  resources.  Although  threats  to  the  nation’s  air 
sovereignty  continue  to  emerge  and  evolve,  GAO  found  that  DOD  is  unable  to 
measure  the  extent  to  which  ASA  helps  to  achieve  the  department’s  homeland- 
defense  goal  of  securing  the  United  States  from  direct  attack  because  DOD  has 
not  established  performance  measures.  NORAD  has  not  conducted  routine  risk 
assessments  of  ASA  operations.  DOD  has  also  yet  to  conduct  a  cost-benefit 
analysis  for  two  of  the  three  alternatives  to  current  ASA  operations  that  GAO 
evaluated.  Adopting  a  more-rigorous  risk-based  management  approach — 
including  balancing  risk  and  costs — would  help  policymakers  within  DOD  and 
elsewhere  more  effectively  allocate  finite  DOD  resources. 

Weak  internal  controls  limit  the  ability  of  the  Air  Force  and  National  Guard 
Bureau  to  accurately  identify  ASA  expenditures.  GAO  analyzed  the  fiscal  year 

2010  expenditure  information  that  the  Air  Force  and  National  Guard  Bureau 
submitted  to  Congress  along  with  DOD’s  fiscal  year  2012  budget  justification  and 
found  the  reported  expenditures  of  more  than  $246  million  to  be  inaccurate.  For 
example,  GAO  found  that  the  Air  Force  overstated  ASA  flying-hour  expenditures 
by  at  least  $22  million  and  included  expenditures  related  to  national  special- 
security  events,  which  are  not  part  of  ASA  operations.  GAO  found  that  the  Air 
Force’s  ability  to  identify  ASA  expenditures  is  limited  by  unclear  roles  and 
responsibilities  for  programming  and  budgeting  and  a  lack  of  guidance  on 
defining  and  tracking  ASA  expenditures.  These  types  of  internal  controls  are 
important  to  ensuring  basic  accountability,  maintaining  funds  control,  and 
preventing  fraud  and  abuse. 
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January  31,  2012 
Congressional  Requesters 

Since  the  September  1 1 ,  2001 ,  terrorist  attacks,  the  U.S.  government  has 
taken  extensive  efforts  to  protect  the  aviation  sector  and  airspace  over 
the  United  States — including  the  establishment  of  restricted  flight  zones; 
installation  of  hardened  cockpit  doors;  enhanced  screening  of  airline 
passengers  and  baggage  before  flights;  increased  use  of  air  marshals  on 
selected  flights;  the  vigilance  of  civilian  passengers  and  flight  attendants 
aboard  commercial  aircraft;  and  having  fully  fueled,  fully  armed  jets  ready 
to  scramble  at  any  time  in  response  to  airborne  threats.  As  a  result  of 
these  efforts,  the  Transportation  Security  Administration  believes  that  a 
hijacking  scenario  over  the  United  States  remains  a  possibility  although  it 
is  less  likely  to  occur  today.  Other  threats  to  the  nation’s  air  sovereignty 
continue  to  emerge  and  evolve.1  As  a  part  of  the  federal  government’s 
overall  efforts  to  protect  U.S.  airspace,  the  Department  of  Defense  (DOD) 
performs  Operation  Noble  Eagle,  which  consists  of  several  operations, 
including  the  maintenance  of  fighter  aircraft  and  trained  personnel  on  alert 
24  hours  a  day,  365  days  a  year,  at  18  alert  sites  across  the  United 
States.2  Since  its  inception  in  2003,  these  efforts  have  been  commonly 
referred  to  as  air  sovereignty  alert  (ASA)  operations.  The  ASA  sites  serve 
to  deter,  respond  to,  and  if  necessary,  defeat  airborne  threats  over  the 
United  States  and  Canada.  In  effect,  these  fighter  jets  can  be  the  last  line 
of  multiple  layers  of  defense. 

As  a  component  of  Operation  Noble  Eagle,  ASA  operations  are 
conducted  by  several  DOD  components  under  the  command  of  North 


Airborne  threats  to  the  nation  can  come  from  a  range  of  vehicles  (e.g.,  commercial, 
general  aviation,  cargo,  ultra  light,  and  military  aircraft  and  cruise  missiles)  and  modes  of 
attack  (e.g.,  detonating  explosives  on  board,  hijacking,  crashing  the  aircraft,  and  launching 
of  weapons). 

2Operation  Noble  Eagle,  which  was  initiated  after  the  terrorist  attacks  of  September  1 1 , 
2001,  is  DOD’s  umbrella  homeland  defense  operation  for  North  America  and  Hawaii. 
There  are  a  variety  of  missions  that  could  be  conducted  underneath  this  umbrella 
operation.  For  example,  North  American  Aerospace  Defense  Command  (NORAD)  could 
conduct  irregular  air  patrols  above  major  metropolitan  areas  and  critical  infrastructure 
facilities  as  well  as  maintain  an  alert  force  of  fighter,  tanker,  and  control  aircraft.  Aircraft 
sorties  and  alert  commitments  are  based  on  a  tiered  response  system.  As  threat  levels 
intensify,  the  number  of  aircraft  on  alert  and  on  patrol  increases. 
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American  Aerospace  Defense  Command  (NORAD)  for  the  airspace  over 
North  America,  which  includes  Alaska  and  the  continental  United  States, 
and  under  the  command  of  U.S.  Pacific  Command  for  the  airspace  over 
the  state  of  Hawaii  and  U.S.  territories  in  the  Pacific.  The  Joint  Chiefs  of 
Staff  have  directed  the  Air  Force  (among  other  components)  to  provide 
support  for  ASA  operations.  Generally,  Air  National  Guard  aircraft  and 
personnel  are  used  for  alert,  since  the  National  Guard  Bureau  believes 
that  ASA  operations  are  consistent  with  the  Air  National  Guard’s 
homeland  mission.  Both  the  Air  Force’s  Air  Combat  Command  and  the 
National  Guard  Bureau3  believe  that  Air  National  Guard  units  can  conduct 
alert  duty  with  less  effect  on  overseas  missions  and  at  a  lower  cost  than 
active-duty  Air  Force  units.  Consequently,  Air  National  Guard  units  are  on 
alert  at  1 7  of  the  1 8  ASA  sites.  In  addition,  units  at  all  1 8  ASA  sites  (i.e. , 
both  Air  National  Guard  and  active-duty  personnel)  are  dual-tasked  to 
conduct  both  expeditionary  missions,  such  as  Operation  Enduring 
Freedom  in  Afghanistan,  and  ASA  operations. 

In  January  2009,  we  issued  a  report  that  identified  a  number  of 
shortcomings  in  DOD’s  management  of  ASA  operations.4  First,  we  found 
that  NORAD  faced  difficulty  determining  the  appropriate  levels  and  types 
of  units,  personnel,  and  aircraft  for  ASA  operations.  We  also  found  that, 
after  8  years  of  operations,  the  Air  Force  had  not  yet  established  ASA  as 
a  steady-state  mission,  which  would  include  integrating  ASA  operations 
into  the  Air  Force’s  planning,  programming,  and  budgeting  cycle.  Partly 
resulting  from  the  absence  of  a  steady-state  mission,  NORAD  had  only 
partially  assessed  the  readiness  of  units  that  consistently  conduct  ASA 
operations  to  ensure  that  those  units  were  organized,  trained,  and 
equipped  to  perform  ASA  operations.  In  addition,  we  found  that  the  Air 
Force  faced  significant  challenges  replacing  or  extending  the  service  life 
of  aging  fighter  aircraft,  to  the  extent  that,  if  aircraft  were  not  replaced,  1 1 


3The  National  Guard  Bureau  is  a  joint  function  within  the  DOD  that,  among  other  things, 
participates  with  staffs  from  the  Department  of  Army  and  the  Department  of  Air  Force  in 
the  formulation,  development,  and  coordination  of  all  programs,  policies,  concepts,  and 
plans  pertaining  to  or  affecting  the  National  Guard,  the  Army  National  Guard  of  the  United 
States,  and  the  Air  National  Guard  of  the  United  States.  The  Air  National  Guard  of  the 
United  States  is  a  reserve  component  of  the  U.S.  Air  Force. 

4GAO,  Homeland  Defense:  Actions  Needed  to  Improve  Management  of  Air  Sovereignty 
Alert  Operations  to  Protect  U.S.  Airspace,  GAO-09-184  (Washington,  D.C.:  Jan.  27, 
2009).  In  addition,  GAO  testified  on  this  topic  in  April  2009,  GAO,  Homeland  Defense: 
Actions  Needed  to  Address  Management  of  Air  Sovereignty  Alert  Operations  to  Protect 
U.S.  Airspace,  GAO-09-612T  (Washington,  D.C.:  Apr.  22,  2009). 
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of  the  18  ASA  sites  could  be  without  aircraft  by  2020.  On  the  basis  of 
those  findings,  we  identified  a  number  of  actions  that  DOD  needed  to  take 
to  improve  management  of  ASA  operations.  Specifically,  we 
recommended  that  DOD  should  conduct  routine  risk  assessments  as  part 
of  a  risk-based  management  approach,  implement  ASA  as  a  steady-state 
mission,  and  develop  plans  to  address  future  challenges. 

GAO  and  other  federal  agencies  have  long  advocated  the  use  of  a  risk- 
based  management  approach  to  enhance  preparedness  against  potential 
terrorist  threats.5  Risk  management  is  a  systematic,  analytical  process  to 
determine  the  likelihood  that  a  threat  will  harm  physical  assets  or 
individuals  and  then  to  identify  actions  to  reduce  risk  and  mitigate  the 
consequences  of  an  attack.  The  principles  of  risk  management 
acknowledge  that  while  risk  generally  cannot  be  eliminated,  enhancing 
protection  from  known  or  potential  threats  can  serve  to  reduce  risk.  Key 
elements  of  risk  management  include  assessing  threats,  vulnerabilities, 
and  criticality  of  assets  and  selecting  between  alternative  courses  of 
action  to  mitigate  risk.  As  we  have  previously  reported,  risk  management 
can  help  policymakers  make  informed  decisions,  take  actions,  and 
prioritize  resource  investments  under  conditions  of  uncertainty.  For 
additional  information,  see  the  Related  GAO  Products  list  at  the  end  of 
this  report. 

On  the  basis  of  your  request  that  we  review  DOD’s  actions  on  our 
January  2009  recommendations,  other  issues  associated  with  ASA 
operations,  and  the  Air  Force’s  ability  to  sustain  those  capabilities,  this 
report  examines  the  extent  to  which  (1)  the  Air  Force  has  implemented 
the  recommendations  made  to  the  service  in  our  2009  report,  (2)  DOD 
has  implemented  a  risk-based  management  approach  for  ASA 
operations — including  our  2009  recommendation  for  routine  risk 
assessments — that  balances  risk  with  cost,  and  (3)  the  Air  Force 
accurately  identified  expenditures  for  its  ASA  operations.  We  are  also 
issuing  a  classified  version  of  this  report.  That  version  includes  a 
classified  appendix  of  our  analysis  of  NORAD’s  ASA  basing  model. 


5GAO,  Homeland  Security:  Key  Elements  of  a  Risk  Management  Approach, 

GAO-02-1 50T  (Washington,  D.C.:  Oct.  12,  2001);  Intelligence,  Surveillance,  and 
Reconnaissance:  DOD  Needs  a  Strategic,  Risk-Based  Approach  to  Enhance  Its  Maritime 
Domain  Awareness,  GAO-1 1-621  (Washington,  D.C.:  June  20,  201 1 );  Joint  Strike  Fighter: 
Strong  Risk  Management  Principles  Essential  as  Program  Enters  Most  Challenging 
Phase,  GAO-09-71  IT  (Washington,  D.C.:  May  20,  2009);  DOD  Instruction,  3020.45, 
Defense  Critical  Infrastructure  Program  (DCIP)  Management  (Apr.  21 , 2008). 
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To  determine  the  extent  to  which  the  Air  Force  has  implemented  the 
recommendations  from  our  2009  report,  we  reviewed  ASA-related 
guidance,  plans,  and  other  documents  from  the  Office  of  the  Assistant 
Secretary  of  Defense  for  Homeland  Defense  and  Americas’  Security 
Affairs,  Air  Force’s  homeland  defense  office,  Air  Force’s  Air  Combat 
Command,  National  Guard  Bureau,  and  3  of  the  18  ASA  units.  We  also 
interviewed  officials  from  these  organizations.  To  determine  the  extent  to 
which  DOD  has  adopted  a  risk-based  management  approach  to  balance 
risk  and  costs  for  ASA  operations,  we  evaluated  documents  and 
interviewed  officials  from  NORAD,  the  Office  of  the  Assistant  Secretary  of 
Homeland  Defense  and  Americas’  Security  Affairs,  the  Air  Force,  and  the 
National  Guard  Bureau.  On  the  basis  of  our  analyses  of  these  documents 
and  discussions,  we  compared  actions  that  DOD  has  taken  to  three  of  the 
five  phases  within  GAO’s  risk-based  management  framework.6  To 
determine  the  extent  to  which  the  Air  Force  was  able  to  accurately  identify 
ASA  expenditures,  we  analyzed  the  fiscal  year  2010  expenditure 
information  the  Air  Force  included  in  ASA  budget  justification  materials 
provided  to  Congress  in  support  of  the  fiscal  year  2012  budget  and 
documented  the  steps  taken  to  identify  those  expenditures  through 
interviews,  e-mails,  and  related  documentation.  We  also  compared  the 
practices  used  by  the  Air  Force  to  prepare  the  ASA  budget  justification 
against  best  practices  for  producing  cost  estimates.7  To  determine  the 
expenditures  of  each  ASA  site,  we  developed  a  survey  that  we  sent  out  to 
the  Air  National  Guard  and  active-duty  Air  Force  ASA  units  to  gather 
fiscal  year  2009  and  2010  expenditures.  We  were  provided  information 
from  18  of  the  19  ASA  sites  that  incurred  expenditures  for  ASA 


6We  could  not  evaluate  the  extent  to  which  DOD  implemented  the  last  two  phases — 
management  selection  and  implementation  and  monitoring — because  (1)  DOD  had  not 
fully  implemented  the  first  three  phases  of  GAO’s  risk-based  management  framework, 
which  are  essential  for  the  latter  two  phases,  and  (2)  DOD  was  still  in  the  decision-making 
stage  for  the  alternatives  that  we  identified. 

7GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs,  GAO-09-3SP  (Washington,  D.C.:  March  2009). 
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operations.8  Of  the  18  sites  that  provided  us  information,  3  of  the  sites 
provided  us  incomplete  information.9 

We  conducted  this  performance  audit  from  March  2011  to  January  201 2 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  Additional  information  on  our 
scope  and  methodology  appears  in  appendix  I. 


Background 


ASA  Operations  ASA  operations  consist  of  ground  operations  that  take  place  before 

fighter  aircraft  take  off,  including  such  activities  as  maintaining  the  fighter 
aircraft.  They  also  include  those  activities  that  may  take  place  after  a  unit 
receives  an  alert  from  NORAD,  but  before  the  aircraft  are  airborne.  For 
example,  pilots  and  maintenance  personnel  may  rush  from  their  nearby 
lodging  facility  to  the  alert  aircraft  facility,  where  maintenance  personnel 
conduct  final  preparations  while  the  pilots  sit  in  their  aircraft  awaiting 
further  instruction  (battle  station).  Alternatively,  pilots  may  taxi  the  aircraft 
to  the  end  of  the  runway  and  await  further  instruction  (runway  alert)  or 
take  off  in  response  to  the  alert  (scramble). 

Once  aircraft  take  off,  an  Air  National  Guard  pilot  converts  from  Title  32 
status  under  the  command  and  control  of  the  state  governor  to  federal 


8There  are  18  steady-state  ASA  sites.  However,  in  fiscal  year  2010,  McEntire  Joint 
National  Guard  Base,  South  Carolina,  constructed  and  upgraded  facilities  since  it  was 
going  to  take  on  alert  duty  from  Shaw  Air  Force  Base,  South  Carolina,  at  the  beginning  of 
fiscal  year  2011.  Therefore,  we  included  McEntire  Joint  National  Guard  Base’s  fiscal  year 
2010  expenditures  in  our  survey  efforts.  Budget  officials  from  Elmendorf  Air  Force  Base, 
Alaska,  told  us  that  they  were  unable  to  provide  us  accurate  ASA  expenditure  information 
because  the  base  does  not  track  ASA  expenditures  separately  from  its  other  mission 
expenditures. 

9Budget  officials  from  Shaw  Air  Force  Base,  South  Carolina;  Homestead  Air  Force  Base, 
Florida;  and  Hickam  Air  Force  Base,  Hawaii,  were  unable  to  provide  us  information  for 
some  aspects  of  the  operation,  such  as  flying  hour  or  personnel  expenditures. 
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Title  10  status  under  the  command  and  control  of  NORAD.10  Within  limits, 
NORAD  can  increase  personnel,  aircraft,  and  the  number  of  ASA  sites 
based  on  changes  in  the  threat  conditions. 


Risk-Based  Management  GAO  has  previously  reported  that  a  risk-based  management  approach 
Approach  helps  policymakers  make  informed  decisions  and  prioritize  resource 

investments.  Risk  management  is  a  widely  endorsed  strategy  for  helping 
decision  makers  allocate  finite  resources  and  take  action  under  conditions 
of  uncertainty.  We  have  previously  recommended  a  five-phase  risk-based 
management  approach  as  shown  in  figure  1. 


Figure  1:  GAO’s  Risk-Based  Management  Framework 


Strategic  goals,  objectives,  and  constraints 

Addresses  what  the  strategic  goals  are  attempting  to  achieve  and  the  steps 
needed  to  attain  those  results. 

Risk  assessment 

Addresses  identification  of  key  elements  of  potential  risks  so  that 
countermeasures  can  be  selected  and  implemented  to  prevent  or 
mitigate  their  effects. 

Alternatives  evaluation 

Addresses  the  evaluation  of  alternative  countermeasures  to  reduce  risk  being 
considered  with  associated  costs 

Management  selection 

Addresses  where  resources  and  investments  will  be  made  based  on 
alternatives  evaluation  and  other  management  criteria,  such  as  availability  of 
funds. 

Implementation  and  monitoring 

Addresses  how  countermeasures  will  be  applied  and  the  mechanism  to  keep 
security  measures  updated 


Source  GAO 


The  framework  is  dynamic  and  new  information  can  be  entered  at  any 
phase.  The  framework  can  be  used  to  inform  agency  officials  and 


10Title  32  and  Title  10  refer  to  sections  of  the  United  States  Code. 
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decision  makers  of  the  basic  components  of  a  risk-based  management 
approach  or  can  be  used  as  a  stand-alone  guide.  The  risk-based 
management  approach,  as  outlined  above  is  designed  to  be  flexible  in 
that  the  approach  may  be  applied  at  various  organizational  levels  from  a 
department  or  a  multiagency  organization  down  to  specific  projects  or 
operations,  such  as  ASA  operations. 

As  we  previously  reported,  because  there  is  no  one  uniformly  accepted 
approach  to  risk  management,  terms  and  activities  may  differ  across 
applications.  For  example,  the  Department  of  Homeland  Security’s  2009 
National  Infrastructure  Protection  Plan  identifies  a  six-phase  risk- 
management  framework  as  shown  in  figure  2. 11  The  overall  difference 
between  this  framework  and  GAO’s  is  that  the  National  Infrastructure 
Protection  Plan’s  framework  splits  GAO’s  risk-assessment  phase  into  two 
separate  phases. 


Figure  2:  National  Infrastructure  Protection  Plan’s  Risk-Management  Framework 


Assets,  systems, 
and  networks 

Physical 
Cyber 
Human 


Set 

security 

goals 


Identify 

assets, 

systems, 

networks, 

and 

functions 


Assess  risks 

(consequences, 
vulnerabilities, 
and  threats) 


i 

I 


V  V  V  V  V  V 


Continuous  feedback 


Source:  Department  of  Homeland  Security 


According  to  the  2009  National  Infrastructure  Protection  Plan,  the  core 
criteria  for  risk  assessments  include  information  needed  to  understand 
and  address  each  of  the  following  components  of  risk — threat, 
vulnerability,  and  consequence.  Any  approach  that  omits  the  substance  of 
the  steps  may  result  in  resources  that  are  not  targeted  to  the  highest 


^Department  of  Homeland  Security,  National  Infrastructure  Protection  Plan  (2009). 
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security  needs.  In  addition,  failing  to  monitor  the  implementation  of 
countermeasures,  including  those  implemented  by  other  agencies,  may 
result  in  a  misallocation  of  resources.  Similarly,  failing  to  conduct  routine 
or  periodic  assessments  of  programs  or  operations  could  result  in  missed 
opportunities  to  increase  their  efficiency  and  effectiveness. 

DOD,  itself,  has  identified  risk  management,  in  this  context,  as  a  process 
of  systematically  identifying,  assessing,  and  managing  risks  arising  from 
operational  factors  and  making  decisions  that  balance  cost  with  mission 
benefits.  For  example,  DOD’s  defense  critical  infrastructure  program  is  a 
risk-management  program  that  seeks  to  ensure  the  availability  of  certain 
identified  assets  critical  to  DOD  missions.  Activities  include  the 
identification,  assessment,  and  security  enhancement  of  assets  essential 
for  executing  the  National  Military  Strategy.  The  Defense  Logistics 

Agency  similarly  established  its  enterprise  risk-management  approach  to 
identify,  assess,  communicate,  and  manage  risk  in  a  cost-effective 
manner. 

The  Air  Force  Has 

Not  Fully 

Implemented  the 
Recommendations 
from  Our  2009  Report, 
and  Mission 

Confusion  Persists 

The  Air  Force  has  fully  implemented  one  recommendation  from  our  2009 
report,  but  has  not  fully  implemented  the  remaining  four 
recommendations.  In  addition,  confusion  over  the  definition  and  scope  of 
the  mission  perpetuates  confusion  about  the  capabilities  and 
expenditures  associated  with  the  mission. 

Air  Force  Has  Taken 

Partial  Steps  to  Implement 
Recommendations 

As  of  November  201 1 ,  the  Air  Force  had  fully  implemented  one  of  the  five 
recommendations  from  our  2009  report12  and  taken  partial  steps  to 
implement  the  remaining  four  recommendations,  as  shown  in  figure  3 
below.  We  made  the  following  recommendations:  (1 )  to  formally  assign 
ASA  duties  to  the  appropriate  units;  (2)  to  implement  ASA  as  a  steady- 
state  mission;  (3)  to  establish  a  timetable  for  implementation  of  the 
preceding  recommendation;  (4)  to  develop  and  implement  a  plan  to 

12GAO-09-184. 
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address  any  projected  capability  gaps;  and  (5)  to  develop  and  implement  a 
formal  method  to  replace  deploying  units. 


Figure  3:  Air  Force’s  Implementation  of  Our  2009  Recommendations  as  of  November  2011 


Interactivity  instructions 

Click  on  a  recommendation  to  see  the  actions  taken.  Click  on  the  “x"  to  clear.  I  I  See  appendix  II  for  the  non-interactive,  printer-friendly  version. 


Recommendation 


Grade 


Actions  taken 


We  recommend  that  the  Secretary  of  Defense  direct  the 
military  services  with  units  that  consistently  conduct  ASA 
operations  to  formally  assign  ASA  duties  to  these  units  and 
then  ensure  that  the  readiness  of  these  units  is  fully 
assessed,  to  include  personnel,  training,  equipment,  and 
ability  to  respond  to  an  alert. 

1 _ i _ i _ i _ 1 

0  50  100 

We  recommend  that  the  Secretary  of  Defense  direct  the 
Secretary  of  the  Air  Force  to  implement  ASA  as  a  steady- 
state  mission  according  to  NORAD,  DOD,  and  Air  Force 
guidance  by: 

(1 )  incorporating  ASA  operations  within  the  Air  Force 
submissions  for  the  6-year  Future  Years  Defense  Program; 

(2)  updating  the  Air  Force  homeland  defense  policy, 
homeland  operations  doctrine,  and  concept  of  operations  to 
incorporate  and  define  the  roles  and  responsibilities  for  ASA 
operations;  and 

(3)  updating  and  implementing  the  ASA  program  action 
directive. 

i _ i _ i _ i _ i 

0  50  100 

We  recommend  that  the  Secretary  of  Defense  direct  the 
Secretary  of  the  Air  Force  to  establish  a  timetable  to 
implement  ASA  as  a  steady-state  mission  [by  taking  the 
actions  identified  in  the  preceding  recommendation]. 

i _ i _ i _ i _ i 

0  50  100 

We  recommend  that  the  Secretary  of  Defense  direct  the 
Secretary  of  the  Air  Force  to  develop  and  implement  a  plan 
to  address  any  projected  capability  gaps  in  ASA  units  due  to 
the  expected  end  of  the  useful  service  lives  of  their  F-15  and 
F-16  fighter  aircraft. 

^^7/  i  i  i 

0  50  100 

We  recommend  that  the  Secretary  of  Defense  direct  the 
Secretary  of  the  Air  Force  to  develop  and  implement  a  formal 
method  to  replace  deploying  units  that  still  provides  unit 
commanders  flexibility  to  coordinate  replacements. 

K/  ///x  i  i  1 

0  50  100 

~|  Recommended  action  complete 
Recommended  action  underway 


Source:  GAO  analysis  of  DOD  information. 
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We  continue  to  believe  that  the  prior  recommendations  have  merit  and 
that  by  not  fully  addressing  the  recommendations,  DOD  could  leave  the 
long-term  sustainability  of  ASA  operations  at  risk.  Since  the  2009  report 
was  issued,  some  issues  have  arisen  that  might  have  been  averted  had 
the  Air  Force  implemented  the  recommendations.  For  example,  since  the 
Air  Force  canceled  the  ASA  program  action  directive  that  clearly  identified 
the  service’s  Air  Combat  Command  as  responsible  for  programming  for 
ASA  operations,  the  National  Guard  Bureau  assumed  programming  and 
budgeting  for  its  ASA  sites  to  ensure  that  its  units  that  were  conducting 
ASA  operations  were  funded.  This  in  turn  has  led  to  the  National  Guard 
Bureau  having  to  fund  emergent  expenses  with  funds  that  were 
programmed  for  other  purposes.  For  example,  according  to  National 
Guard  Bureau  officials,  when  an  ASA  site  is  undergoing  certain 
unplanned  construction,  the  National  Guard  Bureau  must  fund  the 
transfer  of  ASA  personnel  and  equipment  to  a  separate  base. 


Undefined  New 
Terminology  Regarding 
ASA  Operations 
Perpetuates  Confusion 
about  the  Mission 


Further  complicating  DOD’s  management  approach  to  ASA  operations, 
we  found  considerable  confusion  about  the  capabilities  and  expenditures 
associated  with  ASA  operations.  In  September  2011,  NORAD  stopped 
using  the  term  “air  sovereignty  alert”  (ASA)  and  created  a  new  term, 
“aerospace  control  alert”  (ACA)  without  clearly  defining  ACA  or  the 
capabilities  that  are  now  included  within  it.  When  we  asked  NORAD 
officials  which  capabilities  are  included  under  ACA,  they  told  us  that  ACA 
includes — but  is  not  limited  to — DOD’s  traditional  scope  of  ASA 
operations  (i.e.,  the  dedicated  fighter  aircraft  and  personnel  at  18  steady- 
state  alert  sites  across  the  United  States),  as  well  as  a  more- 
comprehensive  list  of  capabilities  used  in  their  aerospace  control 
mission.13  As  figure  4  shows,  according  to  NORAD,  ACA  capabilities 
include  air  defense  and  air  sovereignty  operations  personnel  and 
equipment  at  command  and  control  centers  at  NORAD  and  elsewhere, 
ground-based  defense  systems,  air  patrols,  airborne  radar,  and  air-to-air 
refuelers — in  addition  to  ASA  fighters  sitting  on  ground  alert.  However,  Air 
Force  and  National  Guard  Bureau  officials  told  us  that  while  they  were 
uncertain  as  to  the  definition  of  ACA,  they  believe  the  new  term  is 


13NORAD  has  three  overall  missions:  aerospace  warning,  aerospace  control,  and 
maritime  warning.  According  to  NORAD’s  most-recent  chartering  agreement,  aerospace 
control  includes  providing  surveillance  and  exercising  operational  control  of  the  airspace 
of  Canada  and  the  United  States. 
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interchangeable  with  ASA  (i.e.,  the  dedicated  fighter  aircraft  and  personnel 
at  18  steady-state  alert  sites  across  the  United  States). 


Figure  4:  Air  Sovereignty  Alert  (ASA)  versus  Aerospace  Control  Alert  (ACA) 


Source:  NORAD  and  DOD. 
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While  NORAD’s  new  terminology  may  better  reflect  its  overall  aerospace- 
control  mission,  without  a  clear  definition  of  what  capabilities  and 
expenditures  are  related  to  ACA — compared  to  ASA — Congress  will  lack 
a  complete  picture  of  the  funding  and  other  resources  needed  for  DOD  to 
conduct  ASA  operations  (i.e.,  the  dedicated  fighter  aircraft  and  personnel 
at  18  steady-state  alert  sites  across  the  United  States).  DOD  is  required 
by  law  to  submit  a  consolidated  budget  justification  document  annually 
that  includes  all  funding  requirements  for  ASA  operations.  As  will  be 
discussed  in  more  detail  later  in  this  report,  even  before  the  change  in 
terminology,  the  Air  Force  and  National  Guard  Bureau  faced  difficulties 
providing  Congress  with  complete  and  accurate  information  regarding 
expenditures  associated  with  ASA  operations.  During  our  review,  officials 
from  NORAD,  the  Office  of  the  Assistant  Secretary  of  Defense  for 
Homeland  Defense  and  Americas’  Security  Affairs,  the  Joint  Chiefs  of 
Staff,  U.S.  Pacific  Command,  the  Air  Force,  and  the  National  Guard 
Bureau  could  not  consistently  define  what  capabilities  and  expenditures 
should  be  considered  part  of  ASA.  Until  DOD  defines  ACA;  identifies  the 
specific  capabilities  that  support  it — including  personnel  and  equipment; 
and  distinguishes  the  place  of  ASA  within  that  mission  for  the  purposes  of 
providing  consistent  information  in  response  to  their  budget-justification 
requirement,  Congress,  NORAD,  and  other  entities  within  DOD  could  be 
making  important  homeland  security  decisions  based  on  inconsistent 
information. 


DOD  Has  Not  Fully 
Implemented  a  Risk- 
Based  Management 
Approach  That 
Balances  Risk  and 
Costs  for  ASA 
Operations 

Risk  Management  Helps 
Balance  Risk  and  Costs 


DOD  has  taken  a  series  of  actions  for  ASA  operations  that  are  consistent 
with  a  risk-based  management  approach;  however,  several  key  actions 
have  yet  to  be  taken  to  fully  implement  a  risk-based  management 
approach  that  would  enable  the  department  to  better  balance  risk  and 
costs. 


According  to  our  prior  work,  an  ongoing  risk-based  management 
approach  is  a  best  practice  that  enhances  an  organization’s  decision 
making,  including  determining  operational  requirements,  and  helps  to 
guide  the  use  of  limited  resources.  We  have  previously  recommended  a 
five-phase  approach  to  a  risk-management  framework,  including: 
strategic  goals  and  objectives,  risk  assessment,  evaluation  of 
alternatives,  management  selection,  and  implementation  and  monitoring. 
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Risk  management  is  a  widely  endorsed  strategy  for  decision  makers  to 
allocate  finite  resources  and  take  action  under  conditions  of  uncertainty. 
For  example,  the  2009  National  Infrastructure  Protection  Plan  states  that 
its  risk  management  framework  integrates  and  coordinates  strategies, 
capabilities,  and  governance  to  enable  risk-informed  decision  making 
related  to  the  nation’s  critical  infrastructure  and  key  resources.  This 
framework  is  applicable  to  threats  such  as  natural  disasters,  manmade 
safety  hazards,  and  terrorism,  although  different  information  and 
methodologies  may  be  used  to  understand  each. 


DOD  Has  Established  and 
Linked  Its  Homeland 
Defense  Strategic  Goal  to 
Its  ASA  Objectives,  but 
Has  Not  Established 
Performance  Measures 


DOD’s  overarching  homeland  defense  goal  is  to  secure  the  United  States 
from  direct  attack,  but  DOD  is  unable  to  measure  the  extent  to  which  ASA 
operations  help  to  achieve  that  goal.  In  its  2005  Strategy  for  Homeland 
Defense  and  Civil  Support,  DOD  linked  homeland  defense  with  several 
objectives  associated  with  ASA  operations — including  deterring, 
detecting,  and  if  necessary,  defeating  airborne  threats.14  While  these 
actions  are  initial  steps  in  the  first  phase  of  a  risk-based  management 
approach,  NORAD  has  not  identified  performance  measures  that  could 
identify  the  extent  to  which  its  ASA  operations  are  progressing  toward 
DOD’s  homeland  defense  goal  and  objectives.15  While  U.S.  Pacific 
Command  has  established  one  type  of  performance  measure  identified  in 
Joint  Chiefs  of  Staff  doctrine  on  joint  planning  (measures  of 
effectiveness),  the  command  has  not  established  the  other  type  of 
performance  measure  (measures  of  performance).  U.S.  Pacific 
Command  officials  also  recognized  that  the  command’s  performance 
measures  may  not  be  comprehensive  since  its  single  ASA  site  does  not 
deal  with  the  same  operational  tempo  as  the  other  17  ASA  sites. 
According  to  Joint  Chiefs  of  Staff  doctrine  on  planning,  continuous 
evaluation  of  operations  against  measures  of  performance  and  measures 
of  effectiveness  to  assess  the  component’s  progress  toward 
accomplishing  a  task  or  achieving  an  objective  is  a  key  component  of 
DOD’s  planning  process.  Measures  of  effectiveness  are  criteria  used  to 
assess  changes  in  system  behavior,  capability,  or  operational 
environment  that  are  tied  to  measuring  the  attainment  of  an  end  state  or 


1 4 

DOD,  Strategy  for  Homeland  Defense  and  Civil  Support  (Washington,  D.C.:  March 
2005). 

15For  purposes  of  this  report,  performance  measures  refer  to  both  measures  of 
effectiveness  and  measures  of  performance. 
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an  objective,  or  creation  of  an  effect.  Measures  of  performance  are 
criteria  to  assess  friendly  actions  that  are  tied  to  measuring  task 
accomplishment.  To  provide  examples  of  performance  measures  that 
could  be  applied  to  ASA  operations,  we  created  potential  measures  of 
effectiveness  and  measures  of  performance  after  analyzing  NORAD  and 
DOD  planning  documents  and  talking  with  NORAD  and  DOD  officials, 
including  ASA  units.  Table  1  lists  some  of  the  potential  measures  we 
developed.  NORAD  and  DOD  officials  agreed  that  these  examples  could 
be  a  viable  start  for  a  more  comprehensive  list  of  performance  measures 
to  evaluate  the  effectiveness  and  performance  of  ASA  operations. 


Table  1:  Nonexhaustive  GAO-Developed  List  of  Potential  Measures  of  Effectiveness 
and  Measures  of  Performance  for  ASA  Operations 

Measures  of  effectiveness 

Measures  of  performance 

Number  of  track-of-interest  infractions 
within  the  Special  Flight  Restricted  Area 
within  a  certain  period  of  time. 

Number  (or  percentage)  of  times  ASA 
aircraft  launch  within  Operation  Noble 

Eagle  requirements. 

Number  and  type  of  aircraft  declared  to  be 
of  interest  within  an  established  Temporary 
Flight  Restricted  Area  within  a  certain 
period  of  time. 

Number  (or  percentage)  of  times  ASA 
aircraft  launch  within  NORAD  established 
requirements. 

Number  of  track-of-interest  aircraft  that 
divert  after  certain  tactics,  techniques,  and 
procedures  are  performed. 

Number  (or  percentage)  of  times  ASA  pilots 
are  able  to  accurately  identify  and 
communicate  track-of-interest  aircraft  tail 
number  to  command  and  control  entities 
within  5  minutes. 

Total  number  of  commercial  and  general- 
aviation  aircraft  declared  to  be  a  track  of 
interest  within  a  certain  metropolitan  area 
within  a  certain  period  of  time. 

Number  of  Alert  Force  Evaluations  failed 
each  year. 

Source:  GAO  analysis  of  DOD  data. 


Measuring  the  status  of  tasks,  effects,  and  objectives  becomes  the  basis 
for  reports  to  senior  commanders  and  civilian  leaders  on  the  progress  of 
the  operation.  NORAD  could  then  advise  the  President  and  the  Secretary 
of  Defense  accordingly  and  adjust  operations  as  required.  When  we 
discussed  the  lack  of  performance  measures  with  DOD  officials,  they 
acknowledged  that  such  information  would  be  helpful  yet  did  not  exist  for 
ASA  operations. 

DOD  officials  pointed  out  that  NORAD  evaluates  each  ASA  unit  generally 
every  20  months.  As  we  previously  reported,  NORAD’s  unit  assessments 
evaluate  the  quality  of  alert  aircraft,  to  include  the  overall  condition  of  the 
aircraft,  and  a  unit’s  ability  to  respond  to  different  air  sovereignty 
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scenarios.16  While  these  assessments  evaluate  the  equipment  and 
operations  of  a  single  ASA  unit  against  a  planned  scenario,  the  individual 
assessments  do  not  provide  NORAD  or  DOD  situational  awareness  of 
actual  ASA  operations  that  occur  every  day  (as  opposed  to  planned 
evaluations);  do  not  allow  either  NORAD  or  DOD  to  identify  trends  across 
ASA  units;  and  do  not  provide  NORAD  or  DOD  criteria  with  which  to 
evaluate  the  effectiveness  and  performance  of  ASA  operations  against 
DOD’s  overall  homeland-defense  goal.  Without  ASA  performance 
measures,  NORAD  and  DOD  are  limited  in  their  ability  to  assess  ASA 
operations.17 


NORAD  Has  Not 
Conducted  Routine  Risk 
Assessments,  and  Its  ASA 
Model  Has  Limitations 


Although  we  have  previously  recommended  that  NORAD  should  conduct 
routine  risk  assessments  of  ASA  operations18 — the  second  phase  of  a 
risk-based  management  approach — we  found  that  it  has  not  implemented 
this  recommendation.  Specifically,  our  2009  ASA  report  stated  that  by 
undertaking  routine  risk  assessments,  NORAD  could  better  evaluate  the 
extent  to  which  previous  threats  have  been  mitigated  by  DOD  or  other 
government  agencies;  better  evaluate  current  and  emerging  threats  to 
determine  which  ones  require  the  most  urgent  attention;  and  determine 
operational  requirements  to  address  changing  conditions.  Moreover, 
routine  risk  assessments  could  help  NORAD  evaluate  alternatives  to 
current  operations,  especially  in  a  resource-restricted  environment. 
Further,  such  assessments  could  enhance  NORAD’s  ability  to  determine 
and  apply  the  appropriate  level  and  type  of  resources — including  units, 
personnel,  and  aircraft — to  ASA  operations. 


Although  NORAD  has  not  conducted  routine  risk  assessments,  it  took 
steps  toward  implementing  our  2009  recommendation  by  developing  a 
computer  model  to  evaluate  current  ASA  basing  locations.  However,  we 
found  that  the  model  had  certain  limitations  that  limit  its  ability  to  provide 
NORAD  commanders,  DOD  decision  makers,  and  Congress  risk-based 
information.  Specifically,  it  did  not  include  a  prioritized  list  of  metropolitan 
areas  and  critical  infrastructure  locations  that  NORAD  should  protect.  The 


16GAO-09-184. 

17According  to  DOD’s  joint  doctrine  on  planning,  an  assessment  with  performance 
measures  should  answer  two  questions:  (1 )  Is  the  Joint  Force  Commander  doing  things 
right?  and  (2)  Is  the  Joint  Force  Commander  doing  the  right  things? 

18GAO-09-184. 
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2009  National  Infrastructure  Protection  Plan  states  that  identifying  and 
prioritizing  key  assets  to  protect,  such  as  metropolitan  areas  and  critical 
infrastructure,  is  a  key  component  for  risk  management.  According  to 
NORAD  officials,  the  model  omitted  this  information  because  DOD  had 
not  issued  additional  guidance  since  the  initial  lists  were  distributed  after 
the  September  1 1 , 2001 ,  terrorist  attacks.  The  model  also  did  not 
incorporate  data  that  are  associated  with  risk:  threat,  vulnerability,  and 
consequence  because  NORAD  designed  the  model  to  assess  the  ability 
of  its  aircraft  to  respond  to  a  single  type  of  threat  regardless  of  its  location 
in  the  United  States. 

We  also  found  some  limitations  with  the  assumptions  and  reliability  of  the 
input  data  used  by  NORAD’s  computer  model.19  NORAD  officials  agreed 
with  our  observations  and  agreed  to  take  action  to  address  them.  Further, 
we  found  that  while  NORAD  officials  regularly  run  queries  using  the 
model,  they  do  not  generate  reports  documenting  the  results  of  their 
analyses.  GAO’s  Standards  for  Internal  Control  in  the  Federal 
Government  recommends  ongoing  monitoring,  in  addition  to  separate 
evaluations,  to  assess  the  quality  of  performance  over  time  and  ensure 
that  the  findings  of  audits  and  other  reviews  are  promptly  resolved.20  A 
report  that  documents  the  model’s  analyses  may  allow  analysts  to 
routinely  identify  trends  over  a  period  of  time  so  that  commanders  could 
determine  the  extent  to  which  their  operations  are  effective  or  to  which  a 
threat  might  be  evolving.  We  believe  that  NORAD’s  computer  model,  if 
properly  designed,  could  sufficiently  address  our  2009  recommendation 
to  conduct  routine  risk  assessments,  as  well  as  allow  NORAD  to  assess 
the  specifics  of  detailed  ASA  scenarios.  These  areas  of  improvement  are 
consistent  with  DOD  methodology  used  in  other  DOD  risk-management 
efforts,  such  as  force  protection  and  defense  critical  infrastructure,  as  well 
as  the  U.S.  Coast  Guard’s  and  the  Transportation  Security 
Administration’s  risk-based  efforts.  Similarly,  The  National  Strategy  for  the 
Physical  Protection  of  Critical  Infrastructures  and  Key  Assets  states  that 
responsible  authorities  using  models  and  simulations  can  evaluate  the 
risk  associated  with  particular  vulnerabilities  more  accurately  and 


1 Q 

Specific  information  about  the  model’s  inputs  and  areas  for  improvement  are  provided  in 
the  classified  version  of  this  report. 

20GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIM D-00-21 .3.1 
(Washington,  D.C.:  November  1999). 


Page  16 


GAO-12-311  Homeland  Defense 


subsequently  make  more-informed  protection  decisions.21  Without 
conducting  routine  risk  assessments  or  maintaining  a  risk-based 
computer  model  that  includes  key  components  identified  above,  DOD  will 
be  unable  to  identify  and  assess  the  level  of  risk  that  it  is  taking;  evaluate 
its  ASA  posture  as  the  threat  evolves  or  is  mitigated  by  other  U.S. 
government  efforts;  and  make  risk-based  decisions,  especially  in  a 
constrained  fiscal  environment.  In  addition,  without  documenting  the 
results  of  risk  assessments  or  the  analyses  that  are  completed  using  data 
from  the  computer  model,  DOD  will  be  limited  in  its  ability  to  identify 
trends  in  threats,  vulnerabilities,  and  potential  consequence  over  time. 


DOD  Has  Considered 
Costs  and  Benefits  of  One 
of  Three  Alternatives  to 
ASA  Operations 


We  identified  three  alternatives  to  DOD’s  current  ASA  operations  that  the 
department  has  considered  since  2009,  and  DOD  has  analyzed  costs  and 
benefits  of  these  alternatives  to  varying  extents.  During  this  review,  we 
found  that  DOD  has  considered  the  following  three  alternatives  to  its 
current  ASA  posture:  establish  a  new  ASA  site  to  support  the  President 
while  in  Chicago;  acquire  additional  capabilities  to  address  a  perceived 
threat;  and  modify  the  current  number  and  location  of  ASA  sites.  Of  these 
three  alternatives,  we  determined  that  DOD  conducted  a  cost-benefit 
analysis  for  one  of  the  alternatives,  but  had  not  conducted  a  cost-benefit 
analysis  on  the  other  two  alternatives  at  the  time  of  our  review.  According 
to  GAO’s  risk-based  management  framework,  weighing  costs  and 
benefits  is  a  critical  component  of  risk  management  when  choosing 
among  alternative  security  measures  because  it  links  the  benefits  of 
alternatives  to  the  costs  associated  with  implementing  and  maintaining 
them.  The  actions  that  DOD  should  take  in  the  “risk-assessment”  phase 
should  be  the  basis  for  identifying  and  considering  alternatives.  Capital 
investments  of  federal  expenditures  generally  require  a  cost-benefit  or 
cost-effectiveness  approach.22  We  analyzed  the  three  alternatives  to  ASA 
operations  that  DOD  officials  told  us  they  had  considered  since  2009: 


21  White  House,  The  National  Strategy  for  the  Physical  Protection  of  Critical  Infrastructures 
and  Key  Assets  (February  2003). 

220ffice  of  Management  and  Budget  (OMB)  circulars  A-1 1  (sect.  7),  A-94,  and  A-130 
generally  apply  to  capital  investments.  For  example,  OMB  Circular  A-94  outlines 
procedures  for  cost-benefit  analysis  that  are  suggested  for  use  in  the  internal  planning  of 
executive-branch  agencies,  but  are  mandatory  in  certain  contexts,  such  as  for  analyses 
submitted  to  OMB  in  support  of  budget  programs  pursuant  to  OMB  Circular  A-1 1 . 
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•  Establish  a  new  ASA  site  to  support  the  President  while  in  Chicago. 
DOD  evaluated  the  possibility  of  establishing  a  new  ASA  site  in  the 
Chicago  metropolitan  area  to  support  the  President’s  visits  to  his 
hometown.  In  evaluating  this  option,  DOD  identified  several  sites  that 
were  considered  and  analyzed  potential  costs  for  each  of  these  sites. 
In  addition,  DOD  analyzed  the  benefits,  such  as  cost  savings,  and 
drawbacks  associated  with  the  various  sites  and  concluded  that  the 
cost  outweighed  the  benefit  of  an  additional  ASA  site  at  this  location. 

•  Acquire  capabilities  to  address  a  perceived  threat.  NORAD  wants 
DOD  to  acquire  a  number  of  capabilities  for  ASA  operations  to 
address  a  capability  gap  that  NORAD  believes  exists  based  on  its 
ASA  computer  model.  While  NORAD  has  identified  some  of  the 
potential  costs  associated  with  one  of  the  new  capabilities,  it  did  not 
include  all  of  the  costs  associated  with  that  capability;  nor  did  it 
include  costs  associated  with  other  capabilities  that  could  be  acquired. 
However,  since  NORAD  requested  these  additional  capabilities 
through  DOD’s  Joint  Capabilities  Integration  and  Development 
System  process,  a  cost-benefit  analysis  is  not  scheduled  to  occur  until 
NORAD’s  proposal  has  been  assessed  and  approved  as  part  of  that 
process.23 

•  Modify  the  number  and  location  of  ASA  sites.  In  2010,  NORAD 
analyzed  its  ASA  basing  strategy  to  identify  whether  it  could  change 
the  number  and  location  of  ASA  sites  without  affecting  its  ability  to 
defend  the  homeland  from  aviation-related  attack.  NORAD’s  analysis 
determined  that  the  probability  of  success  of  an  attack  increased  as 
the  number  of  ASA  sites  decreased.  The  analysis  also  identified  a 
“point  of  diminishing  returns”  below  which  the  number  of  ASA  sites 
could  not  be  further  reduced  without  assuming  an  unacceptable  level 
of  risk.  However,  the  analysis  did  not  identify  potential  cost  savings 
that  could  result  from  eliminating  a  given  number  of  sites.  Should 
NORAD,  DOD,  or  Congress  consider  modifying  the  number  and 
location  of  ASA  sites  in  the  future,  without  an  analysis  that  balances 
both  risk  and  costs,  decision  makers  will  be  unable  to  make  fully 
informed  decisions  about  whether  the  potential  cost  savings  (or 
increase)  warrants  the  corresponding  increase  (or  decrease)  in  risk. 


oo 

The  Joint  Capabilities  Integration  and  Development  System  analysis  process  is  a 
requirements-generation  system  intended  to  prioritize  and  ensure  that  the  most-essential 
needs  of  the  warfighter  are  met.  According  to  this  process,  when  a  material  solution  is 
recommended  to  mitigate  a  capability  gap,  an  analysis  of  alternatives  is  generally 
conducted  that  evaluates  the  performance,  operational  effectiveness,  operational 
suitability,  and  the  estimated  costs  of  an  alternative  to  meet  a  mission  capability. 
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We  could  not  evaluate  the  extent  to  which  DOD  implemented  the  last  two 
phases — management  selection,  and  implementation  and  monitoring — 
because  (1)  DOD  had  not  fully  implemented  the  first  three  phases  of 
GAO’s  risk-based  management  framework,  which  are  essential  for  the 
last  two  phases,  and  (2)  DOD  was  still  in  the  managerial-selection  phase 
for  the  alternatives  that  we  identified. 

While  we  are  encouraged  that  DOD  has  examined  some  alternatives  to 
its  current  ASA  operations,  adopting  a  more-rigorous  risk-based 
management  approach — including  balancing  risk  and  costs — would  help 
decision  makers  within  DOD  and  elsewhere  more-effectively  allocate 
finite  DOD  resources. 


Weak  Internal 
Controls  Limit  the 

Force  and  National 
Guard  Bureau  to 
Accurately  Identify 
ASA  Expenditures 


We  analyzed  the  fiscal  year  2010  expenditure  information  the  Air  Force 
and  National  Guard  Bureau  submitted  to  Congress24  and  found  that  it  did 
not  accurately  identify  all  ASA  expenditures.25  According  to  the  Air 
Force’s  submission  to  Congress,  more  than  $246  million  was  expended 
on  ASA  operations  in  fiscal  year  2010.  While  the  Air  Force  included  all  of 
the  mandated  funding  requirements  categories  in  its  budget-justification 


Air  Force  and  National 
Guard  Bureau  Officials 
Have  Not  Accurately 
Identified  ASA 
Expenditures 


Ability  of  the  Air 


Internal  control  weaknesses  limit  the  Air  Force’s  ability  to  accurately 
identify  expenditures  with  regard  to  Air  Force  management  of  ASA 
operations.  Without  accurate  expenditure  information,  Congress, 
NORAD,  and  DOD  will  not  have  an  accurate  basis  for  decision  making 
and  realistic  budget  formulation. 


24ln  2008,  Congress  mandated  that  for  fiscal  year  2010  and  each  year  thereafter,  the 
Secretary  of  Defense  submit  to  the  President  for  consideration  by  the  President  for 
inclusion  with  the  budget  materials  submitted  to  Congress  a  consolidated  budget 
justification  that  covers  all  programs  and  activities  of  the  ASA  mission  of  the  Air  Force, 
including  military-personnel  expenditures;  flying  hours;  and  any  other  associated  ASA 
mission  expenditures  of  the  Air  Force  and  the  Air  National  Guard. 

25For  the  purpose  of  this  review,  we  evaluated  the  ASA  fiscal  year  2010  expenditures 
included  in  the  fiscal  year  2012  budget-justification  documents  because  at  the  time  of  our 
review  it  was  the  most-recent  budget  justification  reflecting  resources  spent  on  ASA 
operations. 
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document,  we  found  three  areas  in  which  the  expenditures  reported  by 
the  Air  Force  were  inaccurate. 

First,  we  found  that  the  Air  Force  overstated  its  ASA  flying-hour 
expenditures  by  at  least  $22  million.  According  to  Air  Force  officials,  they 
mistakenly  used  a  placeholder  for  the  ASA  flying-hour  expenditure  of 
approximately  $57.6  million,  instead  of  using  the  actual  flying-hour 
expenditure  data.  During  the  course  of  our  review,  the  Air  Force  provided 
us  with  actual  flying-hour  data,  which  showed  that  the  service  expended 
$35.6  million — 38  percent  less  than  reported. 

Second,  the  ASA  budget-justification  document  included  expenditures 
that  were  not  associated  with  ASA  operations.  For  example,  it  included 
flying-hour  and  personnel  expenditures  related  to  non-ASA  homeland- 
defense  missions,  such  as  air  patrols  protecting  the  President  and 
national  special-security  events.  Hence,  it  is  likely  that  the  Air  Force 
further  overstated  its  ASA  flying-hour  expenditures  beyond  the  $22  million 
discussed  above. 

Third,  the  budget  justification  document  the  Air  Force  provided  to 
Congress  did  not  include  some  expenditures  that  are  related  to  ASA. 
While  the  budget  justification  included  one  military  construction  project  for 
a  munitions  storage  unit  at  the  Toledo,  Ohio,  ASA  site,  it  did  not  include 
any  other  expenditures  associated  with  constructing,  maintaining,  and 
upgrading  ASA  facilities  in  fiscal  year  2010.  For  example,  when  asked  if 
the  officials  included  the  construction  expenditures  related  to  establishing 
an  ASA  site  at  McEntire  Joint  National  Guard  Base,  South  Carolina,  in 
fiscal  year  2010,  such  as  constructing  hangers  for  the  alert  fighter  jets 
and  upgrading  crew  quarters,  the  officials  stated  that  these  expenditures 
were  not  included  in  the  ASA  budget  justification.  However,  we  asked 
budget  officials  at  each  site  about  ASA  expenditures  for  fiscal  year  2010 
in  a  survey.  According  to  their  responses  and  additional  information  from 
command  officials,  ASA  sites  as  a  whole  spent  at  least  $3.8  million  in 
facilities  construction,  maintenance,  and  equipment  that  was  not  included 
in  the  budget  justification. 

When  asked  to  identify  the  expenditures  included  in  the  budget 
justification  and  the  process  by  which  the  ASA  expenditures  were 
compiled,  the  Air  Force  and  National  Guard  Bureau  could  not  answer 
many  of  our  questions  regarding  which  detailed  expenditures  were 
included  under  the  expenditure  totals  listed  in  the  budget  justification. 
Since  the  Air  Force  and  National  Guard  Bureau  were  unable  to  accurately 
identify  the  expenditures  to  conduct  ASA  operations,  we  conducted  a 
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survey  of  all  ASA  sites  in  an  effort  to  identify  approximate  expenditures 
for  fiscal  year  2010.  According  to  the  information  obtained  from  this 
survey,  the  expenditures  for  each  unit  to  conduct  ASA  operations  during 
fiscal  year  2010  ranged  from  $2  million  to  $8  million,  with  the  average  site 
expenditure  being  approximately  $5  million. 

In  addition,  Air  Force  and  National  Guard  Bureau  officials  told  us  they  did 
not  have  documentation  or  guidance  outlining  how  the  ASA  expenditures 
were  defined,  identified,  and  compiled  for  the  budget  justification.  DOD  is 
required  by  various  statutes  to  improve  its  financial-management 
processes,  controls,  and  systems  to  ensure  that  complete,  reliable, 
consistent,  and  timely  information  is  prepared  and  responsive  to  the 
financial  information  needs  of  agency  management  and  oversight 
bodies.26  Additionally,  GAO’s  Standards  for  Internal  Control  in  the  Federal 
Government  cite  the  importance  of  developing  detailed  internal-controls 
policies,  procedures,  and  practices  to  ensure  basic  accountability, 
maintain  funds  control,27  and  prevent  fraud  and  abuse.28  Such  efforts 
should  include  clearly  defined  roles  and  responsibilities  and  accurate  and 
complete  guidance  on  defining  and  tracking  ASA  expenditures.  Without 
adequate  internal  controls  in  place,  the  Air  Force’s  ability  to  accurately 
identify  ASA  expenditures  will  be  limited. 


Unclear  ASA  Roles  and 
Responsibilities 
Contributed  to 
Weaknesses  in  the 
Identification  of  ASA 
Expenditures 


The  Air  Force  and  National  Guard  Bureau  lack  clearly  defined 
programming  and  budgeting  roles  and  responsibilities  for  ASA  operations, 
which  in  turn  limited  the  Air  Force’s  ability  to  identify  ASA  expenditures. 
We  found  that  the  Air  Force  and  National  Guard  Bureau  have  not  clearly 
defined  programming  and  budgeting  roles  and  responsibilities  since  the 
Air  Force  canceled  its  2003  ASA  program  action  directive  in  2009.  The 
directive  required  the  service’s  Air  Combat  Command  to  program  for  ASA 
operations.  As  a  result,  Air  Combat  Command  programmed  and 


26These  statutes  include  the  Chief  Financial  Officers  Act  of  1990,  the  Government 
Management  Reform  Act  of  1994,  the  Federal  Financial  Management  Improvement  Act  of 
1996,  and  various  annual  authorization  and  appropriation  act  provisions. 

27Funds  controls  are  controls  over  the  use  of  public  funds,  including  (1 )  using  public  funds 
efficiently  and  effectively  and  for  the  purposes  and  within  the  time  frames  and  amounts 
prescribed  by  law;  (2)  making  payments  to  the  right  parties  in  the  correct  amount  within 
allowable  time  frames  and  recouping  any  improper  payments;  and  (3)  accurately 
recording  and  reporting  on  transactions  and  use  of  public  funds. 

28GAO/AIMD-00-21 .3.1 . 
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budgeted  for  ASA  operations  between  2003  and  2009.  However,  since 
the  directive  was  canceled  and  no  additional  guidance  was  issued,  Air 
Combat  Command  stopped  programming,  planning,  and  budgeting  for 
ASA  operations  at  Air  National  Guard  sites.  National  Guard  Bureau 
officials  told  us  that  they  consequently  assumed  these  responsibilities. 
However,  Bureau  officials  have  questions  about  the  bureau’s  authority  to 
fund  and  conduct  ASA  operations  in  the  manner  in  which  they  are 
currently  conducted.  We  have  requested  the  Air  Force’s  official  position 
regarding  whether  the  Air  National  Guard  should  be  conducting  this 
mission  under  a  Title  10  or  Title  32  status.  As  of  January  2012,  the  Air 
Force  had  not  responded  to  our  inquiry.  We  also  found  other  internal- 
control  weaknesses,  such  as  a  lack  of  oversight  of  funds  expended  for 
ASA  operations.  According  to  a  National  Guard  Bureau  internal-review 
report,  a  lack  of  oversight  and  other  internal-control  weaknesses,  such  as 
poor  guidance,  led  to  improper  compensation  at  several  ASA  sites  that 
are  operated  by  Air  National  Guard  units.29  Without  clearly  defined  roles 
and  responsibilities,  questions  about  who  is  responsible  for  programming, 
planning,  and  budgeting  will  remain;  and  the  ability  to  accurately  identify 
ASA  expenditures  will  be  limited.  In  addition,  the  Air  Force  and  National 
Guard  Bureau  may  remain  vulnerable  to  improper  use  of  limited 
resources. 


The  Air  Force  and  National 
Guard  Bureau’s  Lack  of 
Guidance  on  Defining  and 
Tracking  ASA 
Expenditures  Limited  Its 
Ability  to  Accurately 
Identify  ASA  Expenditures 


Air  Force  and  National  Guard  Bureau  financial-management  officials  told 
us  that  they  did  not  accurately  capture  the  ASA  expenditures  because 
they  lacked  guidance  that  defines  ASA  operations  and  the  process  to 
track  the  associated  expenditures.  First,  as  mentioned  previously  in  the 
report,  there  is  no  agreed-upon  definition  of  ASA  operations  among 
NORAD  and  DOD  components  that  contribute  to  ASA  operations. 
Consequently,  the  financial-management  guidance  does  not  clearly 
define  the  expenditures  that  should  be  associated  with  ASA  operations. 
Instead,  the  guidance  that  is  provided  to  the  Air  Force  by  the  Office  of  the 
Under  Secretary  of  Defense  (Comptroller)  contains  broad  categories  that 
do  not  clearly  state  what  specific  expenditures  should  or  should  not  be 
included  in  the  budget  exhibit.30  For  example,  the  guidance  associated 


pq 

According  to  the  National  Guard  Bureau,  federal  authorities  are  working  to  recover  the 
improper  compensation. 

on 

The  Office  of  the  Under  Secretary  of  Defense  (Comptroller)  could  not  find  the  original 
ASA  budget-justification  guidance  and  template  so  it  recreated  the  template  for  the 
purposes  of  our  audit. 
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with  identifying  ASA  military-personnel  expenditures  states  that  Air  Force 
officials  should  provide  military-personnel  costs  associated  with  the 
military  full-time  equivalents  supporting  the  ASA  mission.  Since  the 
guidance  does  not  clearly  identify  the  type  or  group  of  military  personnel 
who  are  associated  with  ASA,  financial  managers  could  respond  to  this 
reporting  requirement  with  a  range  of  different  personnel,  such  as 
personnel  that  work  at  NORAD’s  air-defense  sectors  and  DOD’s  refueling 
units  since  they  assist  in  ASA  operations,  but  are  not  necessarily  included 
within  the  18  alert  sites. 

Second,  Air  Force  and  National  Guard  Bureau  financial-management 
officials  told  us  that  they  had  not  issued  any  guidance  explaining  the 
process  for  tracking  ASA  expenditures  at  the  local  level.  According  to  Air 
Force  guidance,  financial  managers  and  budget  officers  at  all  levels  must 
make  sure  that  operating  personnel  are  made  aware  of  the  importance  of 
properly  identifying  and  capturing  all  costs  associated  with  emergency 
and  special  programs,  like  ASA,  because  only  accurate  cost  data  can 
provide  the  documentation  to  meet  reporting  requirements.  Guidance  to 
the  local  level  on  the  process  of  tracking  ASA  expenditures  would  be 
important  because  Air  Force  and  National  Guard  Bureau  officials  told  us 
that  they  develop  the  ASA  budget  justification  using  the  information  that 
the  financial  managers  at  the  ASA  sites  enter  into  the  service’s 
accounting  systems.  While  the  National  Guard  Bureau’s  annual  financial- 
management  guidance  does  not  include  any  information  on  tracking  ASA 
expenditures,  we  found  that  it  does  contain  specific  financial- 
management  guidance  for  a  similar  mission.  Specifically,  the  Air  National 
Guard’s  counterdrug  mission  is  similar  to  the  ASA  mission  in  that  they 
both  involve  Guard  personnel  supporting  a  federal  mission  and  have 
specific  accounting  codes  dedicated  to  tracking  their  expenditures.  When 
we  asked  about  this  disparity,  National  Guard  Bureau  financial- 
management  officials  acknowledged  that  there  was  not  a  good  reason  for 
having  not  included  more  guidance  about  tracking  ASA  expenditures  and 
told  us  that  the  bureau  would  address  this  in  the  next  version  of  the 
guidance.  However,  a  new  version  of  the  National  Guard  Bureau’s 
financial-management  guidance  was  subsequently  issued  and  it  did  not 
include  guidance  on  tracking  ASA  expenditures.  National  Guard  Bureau 
financial-management  officials  stated  that  changes  to  the  guidance  will  be 
driven  by  a  number  of  factors,  including  appropriations  law,  questions 
from  their  comptrollers  in  the  field,  and  our  findings.  Without  clear 
guidance  on  defining  and  tracking  ASA  expenditures,  unit  officials  will  be 
unable  to  provide  Air  Force  and  National  Guard  Bureau  officials  accurate 
and  consistent  ASA  expenditure  information. 
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Conclusions 


Since  the  September  1 1 ,  2001 ,  terrorist  attacks,  the  U.S.  government  has 
taken  extensive  efforts  to  protect  the  airspace  over  the  United  States, 
aviation  threat  profiles  have  changed,  and  fiscal  resources  have  become 
more  constrained.  Nevertheless,  NORAD  and  DOD  have  not  fully 
evaluated  how  to  most-effectively  manage  ASA  operations  in  this  current 
environment.  Until  the  Air  Force  fully  implements  the  recommendations 
identified  in  our  January  2009  report,  including  establishing  ASA  as  a 
steady-state  mission  by  updating  Air  Force  homeland-defense  policy  and 
doctrine,  concerns  about  the  long-term  sustainability  of  ASA  operations 
will  remain.  Additionally,  until  DOD  defines  ACA;  identifies  the  specific 
capabilities  that  support  it,  including  personnel  and  equipment;  and 
distinguishes  the  place  of  ASA  within  that  mission  for  the  purposes  of 
meeting  Congress’s  budget-justification  requirement,  Congress,  NORAD, 
and  other  entities  within  DOD  could  be  making  important  homeland- 
security  decisions  based  on  different  definitions  and  a  resulting  variance 
in  understanding  of  what  is  required  to  accomplish  that  mission. 

Further,  in  this  environment,  it  is  critical  that  DOD  makes  basing, 
acquisition,  and  other  decisions  about  ASA  operations  on  the  basis  of 
risk-management  principles  that  balance  risk  and  costs.  These  principles 
include  establishing  performance  measures;  identifying  and  prioritizing 
locations  that  should  be  protected;  utilizing  accurate,  comprehensive,  and 
documented  risk  assessments;  and  conducting  cost-benefit  analyses  for 
alternatives.  With  a  more-rigorous,  risk-based  management  approach, 
DOD  and  congressional  decision  makers  would  be  in  a  better  position  to 
effectively  allocate  resources.  Finally,  unless  the  Air  Force  addresses 
internal-control  weaknesses  that  limit  its  ability  to  identify  ASA 
expenditures,  Congress,  NORAD,  and  DOD  will  not  have  an  accurate 
basis  for  investment  decision  making  and  realistic  budget  formulation. 


Matter  for 

Congressional 

Consideration 


In  order  to  ensure  that  the  Air  Force  is  taking  action  that  addresses  the 
long-term  sustainability  of  ASA  operations,  Congress  may  wish  to 
consider  requiring  the  Secretary  of  the  Air  Force  to  fully  implement  the 
remaining  actions  identified  in  our  2009  report  within  a  time  period  that 
Congress  believes  most  prudent. 


Recommendations  for 
Executive  Action 


To  address  these  issues,  we  recommend  that  the  Secretary  of  Defense 
take  the  following  seven  actions. 

In  order  that  all  DOD  components  may  better  understand  the  mission  they 
have  been  asked  to  perform,  we  recommend  the  Secretary  of  Defense 
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direct  the  Assistant  Secretary  of  Defense  for  Homeland  Defense  and 
Americas’  Security  Affairs,  in  coordination  with  the  Joint  Chiefs  of  Staff,  to 
clearly  define  ASA  operations  and  its  relationship  to  NORAD’s  ACA 
operations. 

In  order  to  implement  a  more-complete  risk-based  management  approach 
that  balances  risk  and  costs  for  ASA  operations,  we  recommend  the 
Secretary  of  Defense  take  the  following  five  actions: 

•  Direct  the  Joint  Chiefs  of  Staff,  in  coordination  with  the  U.S.  element 
of  NORAD  and  U.S.  Pacific  Command,  to  develop  performance 
measures  for  ASA  operations  and  then  use  these  measures  to 
evaluate  the  mission  and  make  adjustments,  as  warranted,  on  the 
basis  of  the  performance-measure  results. 

•  Direct  the  Office  of  the  Assistant  Secretary  of  Defense  for  Homeland 
Defense  and  Americas’  Security  Affairs,  in  coordination  with  the  Joint 
Chiefs  of  Staff,  to  issue  updated  guidance,  which  includes  a  prioritized 
list  of  metropolitan  areas  and  critical  infrastructure  that  NORAD  is 
supposed  to  protect. 

•  Direct  the  U.S.  element  of  NORAD  to  update  its  ASA  computer  model 
to  address  identified  areas  for  improvement  to  include  incorporating 
the  elements  of  risk — threat,  vulnerability,  and  consequence. 

•  Direct  the  U.S.  element  of  NORAD  to  document  the  results  of  its  risk 
assessments  so  that  NORAD  and  DOD  can  identify  trends  over  time. 

•  Identify  the  appropriate  DOD  entities  that  should  conduct  a 
comprehensive  cost-benefit  analysis  of  NORAD’s  ASA  basing 
strategy  and  then  have  those  entities  conduct  such  an  analysis. 

In  order  to  accurately  identify  ASA  expenditures  and  address  other 
internal  control  weaknesses,  we  recommend  that  the  Secretary  of 
Defense  direct  the  Secretary  of  the  Air  Force  and  the  Director  of  the 
National  Guard  Bureau  to  issue  guidance  that 

•  defines  ASA  programmatic  and  budgeting  roles  and  responsibilities; 

•  defines  all  expenditures  that  should  be  identified  as  ASA  expenditures 
in  financial-management  systems;  and 

•  identifies  the  proper  procedures  to  track  ASA  expenditures  in  their 
financial-management  systems. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  this  report,  DOD  stated  that  it  concurred 
with  one  of  our  recommendations  and  partially  concurred  with  the  other 
six  recommendations.  However,  DOD  did  not  consistently  identify 
specific  actions  that  it  would  take  to  implement  the  recommendations  and 
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it  did  not  identify  time  frames  for  implementing  the  recommendations. 
DOD’s  comments  are  reprinted  in  appendix  III.  The  Department  of 
Homeland  Security  and  the  Federal  Bureau  of  Investigation  also  reviewed 
a  draft  of  this  report  but  did  not  provide  any  comments. 

DOD  concurred  with  our  recommendation  to  clearly  define  ASA 
operations  and  its  relationship  to  NORAD’s  ACA  operations  and  stated 
that  all  organizations  involved  in  the  air  sovereignty  mission  would  use 
standard  terminology  and  that  the  new  terminology  would  be  reflected  in 
subsequent  plans,  execute  orders,  and  DOD  publications. 

DOD  partially  concurred  with  our  recommendation  to  develop 
performance  measures  for  ASA  operations,  but  did  not  identify  any 
specific  actions  that  it  would  take  to  implement  the  recommendation.  In 
its  comments,  DOD  stated  that  it  recognizes  the  merit  of,  and  routinely 
employs  performance  measures  that  are  incorporated  into  military  service 
training  requirements.  However,  NORAD  officials  told  us  during  our 
review  that  the  command  had  not  developed  any  performance  measures 
for  ASA  operations.  Consequently,  we  believe  that  the  development  and 
use  of  ASA-related  performance  measures  would  improve  DOD’s  ability 
to  assess  the  overall  effectiveness  of  its  ASA  operations  and  enhance  the 
performance  measures  to  which  DOD  referred  to  in  its  comments. 

DOD  partially  agreed  with  our  recommendation  that  it  issue  updated 
guidance,  which  includes  a  prioritized  list  of  metropolitan  areas  and 
critical  infrastructure  that  NORAD  is  supposed  to  protect.  DOD  stated  that 
the  current  guidance  on  prioritized  metropolitan  areas  and  critical 
infrastructure  was  produced  by  the  National  Security  Staff  and  the  criteria 
used  to  produce  it  are  still  valid.  However,  this  guidance  has  not  been 
updated  since  shortly  after  September  1 1 , 2001 ,  and  DOD  has  produced 
other  similar  lists  within  the  department  that  it  uses  in  some  contexts.  For 
these  reasons,  we  continue  to  believe  that  DOD  should  consider  updating 
existing  guidance  or  developing  new  guidance  concerning  a  prioritized  list 
of  metropolitan  areas  and  critical  infrastructure  locations.  We  anticipate 
that  NORAD  and  DOD  could  seek  additional  guidance  from  the  National 
Security  Staff  if  they  believe  the  Staffs  guidance  needs  to  be  reviewed  or 
updated.  Such  actions  would  better  ensure  that  key  national  security 
organizations  have  consistent  and  up  to  date  expectations. 

DOD  partially  concurred  with  our  recommendation  to  direct  the  U.S. 
element  of  NORAD  to  update  its  ASA  computer  model  to  address 
identified  areas  for  improvement  to  include  incorporating  the  elements  of 
risk — threat,  vulnerability,  and  consequence.  In  its  comments,  DOD 
stated  that  NORAD  determines  mission  risk  by  evaluating  its  computer 
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model  results  (based  on  vulnerability,  threat  capability,  and  targets  of 
consequence)  and  comparing  the  results  to  intelligence  assessments  of 
threat  intent.  However,  our  analysis  of  the  model  found  that  it  did  not 
incorporate  threat,  vulnerability,  and  consequence  (or  criticality)  data 
because  NORAD  designed  the  model  to  assess  the  ability  of  its  aircraft  to 
respond  to  a  single  type  of  threat  regardless  of  its  location  in  the  United 
States.  DOD’s  comments  also  stated  that  NORAD  would  explore 
improvements  to  the  ASA  model  in  support  of  NORAD/NORTHCOM 
decision  making  as  targets  and  threats  evolve  and  capabilities  improve. 
We  continue  to  believe  it  appropriate  that  NORAD’s  future  improvements 
to  the  model  should  include  the  three  components  that  DOD  has  long- 
considered  essential  in  risk  assessments:  threat,  vulnerability,  and 
consequence  (or  criticality).  Such  actions  would  be  consistent  with  DOD’s 
other  efforts  to  protect  its  people,  property,  and  systems  from  attacks.  We 
also  expect  that  future  improvements  to  the  model  will  include  the  specific 
areas  we  identified  in  the  classified  version  of  this  report. 

DOD  partially  concurred  with  the  recommendation  to  direct  the  U.S. 
element  of  NORAD  to  document  the  results  of  its  risk  assessments  so 
that  NORAD  and  DOD  can  identify  trends  over  time.  In  its  response, 

DOD  identified  several  NORAD  studies  or  assessments — including  a 
2009  Collateral  Damage  Study  and  the  2010  ASA  Basing  Study — that  are 
subject  to  trend  analysis.  We  believe  that  NORAD’s  efforts  to  study  or 
analyze  different  aspects  of  its  missions  are  positive  actions.  However, 
while  NORAD  issued  a  one-time  report  to  Congress  in  March  2011  that 
identified  the  results  of  its  2010  ASA  Basing  Study,  a  one-time  study  is 
not  a  trend  analysis  and  the  command  has  not  documented  the  results  of 
its  analyses  of  the  ASA  model  since  that  time.  We  continue  to  believe 
that  DOD  should  document  over  time  the  results  of  analyses  stemming 
from  the  ASA  basing  model  itself.  This  would  provide  a  valuable 
reference  of  the  model’s  inputs  and  outputs  overtime  and  improve  the 
ability  of  current  and  future  leaders  to  identify  and  address  evolving 
threats  within  U.S.  airspace. 

DOD  partially  concurred  with  our  recommendation  that  the  Secretary  of 
Defense  should  identify  the  appropriate  DOD  entities  that  should  conduct 
a  comprehensive  cost-benefit  analysis  of  NORAD’s  ASA  basing  strategy 
and  then  have  those  entities  conduct  such  an  analysis.  The  response 
stated  that  multiple  cost-benefit  analyses  have  been  conducted  on 
NORAD’s  ASA  basing  strategy  by  NORAD,  OSD  Cost  Assessment  and 
Program  Evaluation,  the  Joint  Staff,  the  Department  of  Air  Force,  and  the 
Office  of  the  Assistant  Secretary  of  Defense  for  Homeland  Defense  and 
Americas’  Security  Affairs.  However,  when  we  asked  NORAD  and  DOD 
officials,  including  some  of  those  listed  above,  for  the  cost-benefit 
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analysis  of  its  basing  strategy  (i.e.  potential  cost  savings  that  could  result 
from  eliminating  a  given  number  of  sites),  the  officials  told  us  that  such  an 
assessment  had  not  been  completed.  Further,  when  we  asked  which 
organization  would  be  responsible  for  completing  such  studies,  the 
officials  provided  us  conflicting  views  of  whose  responsibility  it  would  be. 
Nonetheless,  DOD’s  comments  also  recognize  that  such  cost-benefit 
analyses  are  critical  for  NORAD’s  force  posturing.  As  such,  we  believe 
that  NORAD  and  DOD’s  ability  to  demonstrate  its  use  of  these  cost- 
benefit  analyses  in  future  decision  making  efforts  will  help  to  ensure 
effective  mission  capability  consistent  with  the  current  fiscal  environment. 

DOD  partially  concurred  with  our  recommendation  to  direct  the  Secretary 
of  the  Air  Force  and  the  Director  of  the  National  Guard  Bureau  to  issue 
guidance  that  addresses  internal  control  weaknesses  that  kept  the  Air 
Force  from  providing  accurate  ASA  expenditure  information  to  Congress. 
Such  weaknesses  included  unclear  roles  and  responsibilities;  lack  of 
guidance  that  identified  expenditures  that  should  be  identified  as  ASA 
expenditures  in  financial-management  systems;  and  proper  procedures  to 
track  ASA  expenditures  in  their  financial-management  systems.  In  its 
response,  DOD  stated  that  it  believes  that  programmatic  and  budgetary 
roles  are  clearly  understood  and  well  defined  in  DOD’s  planning, 
programming,  budgeting,  and  execution  process.  However,  the  Air 
Force’s  inability  to  provide  Congress  an  accurate  reporting  of  resources 
used  for  ASA  operations  indicates  that  internal  control  weaknesses  exist 
that  have  not  been  mitigated  by  DOD’s  standard  process.  For  example, 
officials  from  the  Air  Force’s  homeland  defense  office,  the  National  Guard 
Bureau,  and  the  Air  Force’s  Combat  Command  could  not  provide  us  any 
documentation  of  which  organization  is  responsible  for  programming  and 
budgeting  for  ASA  operations  since  the  Air  Force  canceled  its  ASA 
program  action  directive  in  2009. 

As  a  result  of  these  weakness,  the  Air  Force  provided  Congress 
inaccurate  information  about  its  fiscal  year  2010  ASA  expenditures — 
ranging  from  overstating  its  flying  hour  costs  by  at  least  38  percent  to 
understating  its  facilities  construction,  maintenance,  and  equipment 
expenditures.  DOD’s  response  did  not  indicate  that  it  would  issue 
guidance  to  address  these  internal  control  weaknesses.  Given  the  current 
fiscal  environment,  DOD  should  identify  and  take  specific  actions 
immediately  to  address  our  findings  so  that  Congress,  NORAD,  and  DOD 
will  have  an  accurate  basis  for  decision  making  and  realistic  budget 
formulation. 
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We  are  sending  copies  of  this  report  to  appropriate  congressional 
committees,  the  Secretary  of  Defense,  the  Commander  of  NORAD,  the 
Secretary  of  the  Air  Force,  the  Commissioner  of  Customs  and  Border 
Protection,  the  Director  of  the  Federal  Bureau  of  Investigation,  the 
Administrator  of  the  Transportation  Security  Administration,  the 
Commandant  of  the  U.S.  Coast  Guard,  and  the  Director  of  OMB.  In 
addition,  the  report  is  available  at  no  charge  on  the  GAO  website  at 
http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
me  at  (202)  512-4523  or  leporeb@gao.gov.  Contact  points  for  our  Offices 
of  Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last 
page  of  this  report.  GAO  staff  who  made  key  contributions  to  this  report 
are  listed  in  appendix  IV. 


Brian  J.  Lepore 

Director,  Defense  Capabilities  and  Management 
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Appendix  I:  Scope  and  Methodology 


To  determine  the  extent  to  which  the  Air  Force  has  implemented  the 
recommendations  from  our  2009  report,  we  interviewed  officials  from  the 
Assistant  Secretary  of  Defense  for  Homeland  Defense  and  Americas’ 
Security  Affairs,  the  Air  Force’s  homeland-defense  office,  the  Air  Force’s 
Air  Combat  Command,  the  National  Guard  Bureau,  and  three  air 
sovereignty  alert  (ASA)  units.  We  gathered  information  about  steps  taken 
related  to  the  recommendations  and  obtained  supporting  documentation, 
when  possible.  In  those  cases  where  the  Air  Force  had  taken  action,  we 
asked  the  officials  about  the  effect  that  the  action  had  on  their  operations. 
In  those  cases  where  the  Air  Force  had  not  taken  action,  we  asked  the 
officials  to  identify  the  rationale  for  lack  of  action  and  the  plan  that  they 
had  in  place — including  time  frames — to  implement  the 
recommendations. 

To  determine  the  extent  to  which  the  Department  of  Defense  (DOD)  has 
adopted  a  risk-based  management  approach  to  balance  risk  and  costs  for 
ASA  operations,  we  reviewed  and  analyzed  applicable  documentation 
and  interviewed  relevant  officials.  Specifically,  we  reviewed  prior  GAO 
reports  that  recommended  organizations  use  a  risk-based  management 
approach  to  manage  risk  and  costs.1  We  reviewed  documents  and 
interviewed  officials  from  the  Office  of  the  Assistant  Secretary  of  Defense 
for  Homeland  Defense  and  Americas’  Security  Affairs,  the  Air  Force,  and 
the  North  American  Aerospace  Defense  Command  (NORAD)  to 
determine  the  extent  to  which  NORAD  has  followed  a  risk-based 
management  approach  for  ASA  operations.  We  reviewed  NORAD’s  ASA 
assessments  and  compared  them  to  the  elements  required  for  a  risk 
assessment.  We  reviewed  acquisition  documentation  to  determine 
whether  the  command  considered  the  cost  and  benefits  when  considering 
alternatives  for  ASA  operations.  During  our  discussions  with  officials,  we 
asked  how  and  whether  they  measure  the  performance  of  ASA 
operations  and  whether  they  consider  cost  for  ASA  operations.  To  help  us 


^AO,  Joint  Strike  Fighter:  Strong  Risk  Management  Principles  Essential  as  Program 
Enters  Most  Challenging  Phase,  GAO-09-71  IT  (Washington,  D.C.:  May  20,  2009); 
Intelligence,  Surveillance,  and  Reconnaissance:  DOD  Needs  a  Strategic,  Risk-Based 
Approach  to  Enhance  Its  Maritime  Domain  Awareness,  GAO-11-621  (Washington,  D.C.: 
June  20,  201 1 );  Defense  Acquisitions:  Managing  Risk  to  Achieve  Better  Outcomes, 
GAO-1 0-374T  (Washington,  D.C.:  May  20,  2009);  Aviation  Security:  DHS  and  TSA  Have 
Researched,  Developed  and  Begun  Deploying  Passenger  Checkpoint  Screening 
Technologies,  but  Continue  to  Face  Challenges,  GAO-10-128  (Washington,  D.C.:  Oct.  7, 
2009);  and  Risk  Management:  Further  Refinements  Needed  to  Assess  Risks  and 
Prioritize  Protective  Measures  at  Ports  and  Other  Critical  Infrastructure,  GAO-06-91 
(Washington,  D.C.  Dec.  15,  2005). 
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better  understand  whether  DOD  incorporates  risk  into  ASA  operations, 
we  reviewed  documentation  on  how  other  agencies  incorporate  risk  into 
their  missions.  We  then  reviewed  documentation  on  NORAD’s  Operation 
Noble  Eagle  Fighter  Basing  Location  Model  and  analyzed  the  model’s 
inputs  and  assumptions.2  We  interviewed  officials  from  NORAD  on  how  it 
operationalizes  the  model  in  support  of  ASA  operations. 

In  order  to  determine  the  extent  to  which  the  Air  Force  was  able  to 
accurately  identify  ASA  expenditures,  we  analyzed  the  fiscal  year  2010 
expenditure  information  the  Air  Force  included  in  its  fiscal  year  2012  ASA 
budget-justification  document.  We  documented  the  steps  taken  to  identify 
and  compile  the  ASA  expenditures  through  interviews,  e-mails,  and 
related  documentation.  We  then  compared  the  information  and  practices 
used  by  the  Air  Force  to  prepare  the  ASA  budget  justification  against 
cost-evaluation  best  practices  consistently  applied  by  organizations 
throughout  the  federal  government.  Specifically,  we  evaluated  the  budget 
justification  to  determine  whether  it  included  all  expenditures  associated 
with  ASA  operations;3  whether  it  contained  few,  if  any,  mistakes;  and 
whether  the  underlying  documentation  and  expenditure  calculations  were 
well  documented  so  that  they  could  be  easily  repeated  or  updated.  To 
determine  the  expenditures  of  each  ASA  site  we  developed  a  survey, 
which  we  sent  out  to  the  Air  National  Guard  and  active-duty  ASA  units  to 
gather  fiscal  year  2009  and  2010  expenditures.  We  developed  the 
expenditure  survey  by  gathering  ASA  operations-requirement 
documentation  that  outlined  the  personnel,  operational,  and  facilities 
requirements  for  conducting  ASA  operations.  In  order  to  ensure  greater 
reliability  in  the  expenditure  information  we  gathered  from  the  ASA  sites, 
we  instructed  the  sites  to  separate  non-ASA  Operation  Noble  Eagle 
expenditures  from  ASA  expenditures;  include  all  of  the  ASA  construction 
and  maintenance  expenditures;  and  to  remove  duplications  in  their 
responses.  We  also  consulted  with  two  Air  National  Guard  and  two  active 
duty  ASA  sites  to  verify  the  accuracy  and  completeness  of  the  survey’s 
instructions,  parameters,  and  expenditure  categories.  We  found  the  data 
gathered  in  this  survey  to  be  sufficiently  reliable  for  our  purposes. 


2The  Operation  Noble  Eagle  Fighter  Basing  Location  Model  is  a  Monte  Carlo  simulation 
that  creates  events  and  evaluates  how  effective  the  response  will  be,  given  a  predefined 
fighter  posture  and  location. 

3We  defined  ASA  operations  as  the  fully  fueled,  fully  armed  aircraft  and  trained  personnel 
on  alert  24  hours  a  day,  365  days  a  year,  and  the  infrastructure,  equipment,  and  supplies 
required  to  maintain  them  at  the  18  ASA  sites  across  the  United  States. 
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Recommendation 

We  recommend  that  the  Secretary  of 
Defense  direct  the  military  services  with 
units  that  consistently  conduct  Air 
Sovereignty  Alert  (ASA)  operations  to 
formally  assign  ASA  duties  to  these  units 
and  then  ensure  that  the  readiness  of 
these  units  is  fully  assessed,  to  include 
personnel,  training,  equipment,  and  ability 
to  respond  to  an  alert. 


We  recommend  that  the  Secretary  of 
Defense  direct  the  Secretary  of  the  Air 
Force  to  implement  ASA  as  a  steady-state 
mission  according  to  NORAD,  Department 
of  Defense  (DOD),  and  Air  Force  guidance 
by 

(1 )  incorporating  ASA  operations  within  the 
Air  Force  submissions  for  the  6-year 
Future  Years  Defense  Program; 

(2)  updating  the  Air  Force  homeland 
defense  policy,  homeland  operations 
doctrine,  and  concept  of  operations  to 
incorporate  and  define  the  roles  and 
responsibilities  for  ASA  operations;  and 

(3)  updating  and  implementing  the  ASA 
program  action  directive. 

We  recommend  that  the  Secretary  of 
Defense  direct  the  Secretary  of  the  Air 
Force  to  establish  a  timetable  to  implement 
ASA  as  a  steady-state  mission  [by  taking 
the  actions  identified  in  the  preceding 
recommendation]. 


Actions  taken 

Fully  Implemented:  The  National  Guard  Bureau,  in  coordination  with  Air  Combat 
Command,  initially  implemented  this  recommendation  by  placing  ASA  operations  on  the 
units’  secondary  mission  statements.3  However,  the  National  Guard  Bureau  told  us  that  it 
does  not  believe  that  the  Secretary  of  Defense  has  formally  assigned  ASA  operations  to 
the  Air  National  Guard  units.  As  of  November  201 1 ,  bureau  officials  told  us  that  they  were 
taking  action  to  remove  the  reference  to  ASA  operations  from  these  statements  because 
they  now  believe  that  they  could  better  address  the  findings  identified  in  the  2009  report 
through  the  Air  Force’s  standard  deployment  process. 

However,  North  American  Aerospace  Defense  Command  (NORAD)  does  not  believe  that 
the  National  Guard  Bureau  should  remove  the  reference  to  ASA  operations  from  the 
units’  secondary  mission  statements.  NORAD  officials  told  us  that  they  do  not  support  the 
bureau’s  position  because  NORAD  experienced  some  challenges  when  it  placed  units 
that  do  not  normally  sit  alert  on  alert  for  the  1 0th  anniversary  of  the  September  1 1 , 2001 , 
terrorist  attacks.  While  our  assessment  gives  the  Air  Force  credit  for  having  taken  action, 
we  are  also  concerned  that  removing  the  reference  to  ASA  operations  from  the  units’ 
secondary  mission  statements  could  have  adverse  effects,  such  as  the  units  not  reporting 
the  extent  to  which  they  are  able  to  conduct  both  their  ASA  operations  and  overseas 
deployments  in  their  appropriate  readiness  systems.  It  is  too  early  to  determine  the 
consequences  of  the  bureau’s  decision,  although  a  bureau  official  acknowledged  that  the 
problems  we  identified  in  our  2009  report  could  reoccur  if  this  new  approach  with  NORAD 
was  not  fully  implemented. 

Partially  Implemented:  The  Air  Force  has  taken  one  of  three  steps  to  implement  ASA  as 
a  steady-state  mission.  Specifically: 

(1 )  The  National  Guard  Bureau  has  incorporated  ASA  operations  in  its  fiscal  year  2013 
Future  Years  Defense  Program. 

(2)  The  Air  Force  had  not  updated  any  of  the  identified  documents.  Consequently,  overall 
roles  and  responsibilities  remain  undefined. 

(3)  Rather  than  updating  and  implementing  the  ASA  program  action  directive,  the  Air 
Force  cancelled  the  program  action  directive.  This  leaves  the  Air  Force  and  the  National 
Guard  Bureau  without  clear  roles  and  responsibilities  for  operational  planning, 
programming,  and  budgeting  for  ASA  operations,  which  GAO  has  identified  as  an  internal 
control  weakness. 


Not  Implemented:  The  Air  Force  has  not  established  a  timetable  to  implement  ASA  as  a 
steady-state  mission. 
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Recommendation 


Actions  taken 


We  recommend  that  the  Secretary  of 
Defense  direct  the  Secretary  of  the  Air 
Force  to  develop  and  implement  a  plan  to 
address  any  projected  capability  gaps  in 
ASA  units  due  to  the  expected  end  of  the 
useful  service  lives  of  their  F-1 5  and  F-1 6 
fighter  aircraft. 


We  recommend  that  the  Secretary  of 
Defense  direct  the  Secretary  of  the  Air 
Force  to  develop  and  implement  a  formal 
method  to  replace  deploying  units  that  still 
provides  unit  commanders  flexibility  to 
coordinate  replacements. 


Minimally  Implemented:  The  Air  Force  has  taken  some  action  that  could  mitigate 
potential  challenges,  if  proactively  managed,  but  has  not  developed  a  single  plan  to 
manage  the  recapitalization  of  the  aircraft  that  conduct  ASA  operations.  Specifically,  the 
service  has  initiated  efforts  to  test  the  effective  service  life  of  its  remaining  F-1 5  and  F-1 6 
fleets.  In  addition,  the  service  has  requested  funding  to  update  both  platforms  with  radar 
and  communication  equipment  that  will  support  ASA  operations.  The  Air  Force  has  also 
identified  the  extent  to  which  it  could  cover  its  alert  sites  if  either  the  F-1 5  or  F-1 6  fleet 
were  unavailable. 

However,  the  potential  grounding  of  ASA  aircraft — such  as  the  grounding  of  F-1 5s  during 
2007  and  F-22s  during  2011,  the  ongoing  delays  in  the  F-35  program,  and  the 
unspecified  fielding  locations  for  the  F-35  aircraft  could  pose  challenges  in  the  future. 
Further,  the  Air  Force  could  have  challenges  if  the  retirement  of  the  current  fleet  of  F-1 5s 
and  F-1 6s  are  not  managed  in  a  manner  that  considers  the  effect  on  ASA  operations.  For 
example,  if  the  Air  Force  were  to  decrease  the  number  of  older  models  of  F-1 6s  in  the 
inventory,  this  could  affect  ASA  operations  since  these  aircraft  are  flown  by  units  that 
conduct  these  operations.  Also,  if  the  Air  Force  retires  the  fleet  of  F-1 5s  and  F-1 6s  with 
the  longest  remaining  effective  service  lives  before  those  with  shorter  service  lives,  the 
service  could  experience  a  shortage  in  fighter  aircraft  available  for  ASA  operations  before 
the  5th-generation  aircraft  are  available. b 

Minimally  Implemented:  In  response  to  this  recommendation,  DOD  noted  that  it  has  a 
global  force-management  system  that  it  uses  to  formally  manage  its  forces  around  the 
world.  However,  the  approach  that  DOD  uses  to  backfill  personnel  and  units  for  ASA 
operations  is  still  managed  on  an  ad  hoc  basis. 

NORAD  and  the  Air  Force  have  developed  some  planning  documents,  such  as  a  plan  for 
backfilling  ASA  units  on  a  large  scale.  For  example,  one  of  the  planning  documents 
identifies  potential  units  that  could  conduct  ASA  operations  at  bases  that  are  equipped 
with  F-1 5s  if  those  aircraft  were  grounded  for  a  period  of  time  as  they  were  in  2008.  In 
addition,  Air  Combat  Command  and  the  National  Guard  Bureau  developed  a 
memorandum  of  understanding  that  identifies  a  process  by  which  Air  National  Guard 
units  will  be  scheduled  to  support  non-ASA  Operation  Noble  Eagle  missions,  such  as 
national  special-security  events,  throughout  the  year,  if  called  upon.  The  agreement 
identifies  a  process  by  which  Air  National  Guard  units  will  be  identified  and  scheduled  to 
support  these  missions,  if  called  upon,  and  the  time  frame  during  which  each  unit  will  be 
ready  to  support  these  missions.  It  also  identifies  Air  Combat  Command  as  the  command 
that  will  be  responsible  for  providing  funds  to  those  units  when  called  upon. 

However,  the  Air  Force  has  not  developed  similar  Operation  Noble  Eagle  planning 
documents  to  replace  ASA  units  that  deploy  to  support  overseas  operations  or  units  that 
are  taken  off  alert  for  a  short  period  of  time  when  they  fail  an  alert  force  evaluation. 
Consequently,  the  National  Guard  Bureau,  in  coordination  with  Air  Combat  Command, 
must  try  to  find  personnel  and  equipment  that  are  available  to  support  these  units.  In 
addition,  since  it  is  unclear  who  is  responsible  for  funding  these  backfill  efforts,  the 
National  Guard  Bureau  must  use  funds  that  were  programmed  for  other  purposes  to 
cover  these  costs. 


Source:  GAO  analysis  of  DOD  information. 

aThe  Air  Force  issues  mission  designed  operational  capabilities  statements  that  identify  the  unit’s 
mission(s)  and  related  requirements  (e.g.,  type  and  number  of  personnel).  The  unit’s  readiness  is 
based  on  these  requirements. 

bFifth-generation  aircraft,  such  as  the  F-22  and  F-35,  have  stealth  characteristics,  fused  sensor  data, 
and  advanced  radars. 
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Note:  GAO  received 
Department  of  Defense 
comments  on  January  27, 
2012. 


ASSISTANT  SECRETARY  OF  DEFENSE 
2600  DEFENSE  PENTAGON 
WASHINGTON,  D.C.  20301-2600 


HOMELAND  DEFENSE 
&  AMERICAS’  SECURITY  AFFAIRS 

Mr.  Brian  Lepore 

Director,  Defense  Capabilities  and  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Lepore: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  final  report,  “(U) 
Homeland  Defense:  Continued  Actions  Needed  to  Improve  Management  of  Air  Sovereignty 
Alert  Operations,”  dated  January  1 1 , 20 1 2  (GAO  Code  35 1 594/GAO- 1 2-3 1 1 ). 

Thank  you  for  the  opportunity  to  comment  on  this  important  GAO  report.  Defense  of  the 
United  States  and  its  citizens  is  and  will  remain  DoD’s  number  one  priority.  Although  DoD 
works  with  many  interagency  partners  on  overall  air  security,  it  has  the  sole  responsibility  to 
deter  and  defeat  airborne  threats  to  the  homeland.  Air  Sovereignty  is  a  critical  component  of  our 
Homeland  Defense  posture,  and  we  look  forward  to  studying  this  report  in  depth  to  identify 
changes  that  could  strengthen  the  air  defense  of  the  United  States. 

We  greatly  appreciate  GAO’s  efforts  to  ensure  DoD’s  official  written  comments  are 
included  despite  the  expedited  release  timeline. 

Our  point  of  contact  for  this  action  is  LtCol  Jeff  Blackwell,  (571)  256-8322,  or 
thomas.blackuellu  osd.mil. 


Sincerely, 


PaulAl.  Stockton 


Enclosure: 

1 .  DoD  Response  to  FAO- 12-311 


o 
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GAO  REPORT  -  DATED  JANUARY  11,  2012 
GAO  CODE  35 1594/GAO- 12-3 11 

“(U)  HOMELAND  DEFENSE:  Continued  Actions  Needed  to  Improve  Management  of 
Air  Sovereignty  Alert  Operations" 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  RECOMMENDATIONS 

RECOMMENDATION  1:  “In  order  that  all  DoD  components  may  better  understand  the 
mission  they  have  been  asked  to  perform,  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Assistant  Secretary  for  Homeland  Defense  and  Americas'  Security  Affairs,  in  coordination 
with  the  Joint  Chiefs  of  Staff,  to  clearly  define  ASA  operations  and  its  relationship  to  NORAD’S 
ACA  operations.” 

DoD  RESPONSE:  Concur.  OSD  will  coordinate  with  the  Joint  Staff, 
NORAD/USNORTHCOM,  and  the  National  Guard  Bureau  (NGB)  to  ensure  standard 
terminology  is  used  by  all  organizations  involved  in  the  Air  Sovereignty  mission  and  is 
reflected  in  subsequent  plans,  execute  orders,  and  DoD  publications.  This 
standardization  will  also  assist  in  implementing  GAO  Recommendation  #3  as  it  relates  to 
reporting  and  accounting  for  this  mission. 

RECOMMENDATION  2:  “In  order  to  implement  a  more  complete  risk-based  management 
approach  that  balances  risk  and  costs  for  ASA  operations,  GAO  recommends  that  the  Secretary 
of  Defense  take  the  following  actions: 

•  “Direct  the  Joint  Chiefs  of  Staff,  in  coordination  with  the  U.S.  element  of  NORAD  and 
U.S.  Pacific  Command,  to  develop  performance  measures  for  ASA  operations  and  then 
use  these  measures  to  evaluate  and  make  adjustments,  as  warranted,  based  on  the 
performance  measure  results.” 

DoD  RESPONSE:  Partially  Concur.  DoD  recognizes  the  merit  of,  and  routinely 
employs  measures  of  performance.  The  Military  Departments  are  tasked  with  preparing 
and  certifying  combat  units  for  use  by  Combatant  Commanders.  Each  Military 
Department  has  long-established  procedures  and  measures  of  performance  to  certify  units 
as  combat  ready.  Combatant  Command-specific  measures  of  effectiveness  and 
performance  deemed  critical  to  mission  execution  are  incorporated  into  Service  training 
requirements. 

•  “Direct  the  Office  of  the  Assistant  Secretary  of  Defense  for  Homeland  Defense  and 
Americas’  Security  Affairs,  in  coordination  with  the  Joint  Chiefs  of  Staff,  to  issue 
updated  guidance,  which  includes  a  prioritized  list  of  metropolitan  areas  and  critical 
infrastructure  that  NORAD  is  supposed  to  protect.” 
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DoD  RESPONSE:  Partially  Concur.  The  National  Security  Staff  (NSS)  produced  the 
current  guidance  on  prioritized  metropolitan  areas  and  critical  infrastructure.  The  criteria 
used  to  produce  the  guidance  are  still  valid.  OSD  welcomes  the  opportunity  to  review 
and  update  the  guidance  as  required. 

•  “Direct  the  U.S.  element  of  NORAD  to  update  its  ASA  computer  model  to  address 
identified  areas  for  improvement  to  include  incorporating  the  elements  of  risk-threat, 
vulnerability,  and  consequence.” 

DoD  RESPONSE:  Partially  Concur.  NORAD  determines  mission  risk  by  evaluating  its 
computer  model  results  (based  on  vulnerability,  threat  capability,  and  targets  of 
consequence)  and  comparing  the  results  to  intelligence  assessments  of  threat  intent. 
NORAD  does,  and  will  continue  to  explore  improvements  to  the  ASA  model  in  support 
of  NORAD/USNORTHCOM  decision  making,  as  targets  and  threats  evolve,  and 
capabilities  improve. 

•  “Direct  the  U.S.  element  of  NORAD  to  document  the  results  of  its  risk  assessments  so 
that  NORAD  and  DOD  can  identify  trends  over  time.” 

DoD  RESPONSE:  Partially  Concur.  NORAD  conducts  numerous  formal  studies/risk 
assessments  (e.g.,  2009  Collateral  Damage  Study,  2010  ASA  Basing  Study)  that  are 
subject  to  both  formal  and  informal  trend  analysis.  DoD  recognizes  the  value  of  this 
trend  analysis  and  will  continue  to  work  with  NORAD  to  understand  current  risk  and  to 
track  changes  over  time. 

•  “Identify  the  appropriate  DOD  entities  that  should  conduct  a  comprehensive  cost-benefit 
analysis  of  NORAD’s  ASA  basing  strategy  and  then  have  those  entities  conduct  such  an 
analysis.” 

DoD  RESPONSE:  Partially  Concur.  Multiple  cost-benefit  analyses  have  been  and  will 
continue  to  be  conducted  on  NORAD’s  ASA  basing  strategy,  as  they  are  critical  to 
NORAD  force  posturing.  Previous  efforts  include  those  led  by  NORAD,  OSD  Cost 
Assessment  and  Program  Evaluation  (CAPE),  the  Joint  Staff,  the  Department  of  the  Air 
Force,  and  the  OASD(HD&ASA). 

RECOMMENDATION  3:  “In  order  to  accurately  identify  ASA  expenditures  and  address 
other  internal  control  weaknesses,  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Secretary  of  the  Air  Force  and  the  Director  of  the  National  Guard  Bureau  to  issue  guidance  that: 

•  Defines  ASA  programmatic  and  budgeting  roles  and  responsibilities: 

•  Defines  all  expenditures  that  should  be  identified  as  ASA  expenditures  in  financial 
management  systems;  and 

•  Identifies  the  proper  procedures  to  track  ASA  expenditures  in  their  financial  management 
systems.” 
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DoD  RESPONSE:  Partially  Concur.  Programmatic  and  budgetary  roles  are  clearly 
understood.  Responsibilities  are  well  defined  and  established  in  the  DoD  Planning, 
Programming,  Budgeting,  and  Execution  (PPBE)  process.  The  Secretary  of  the  Air  Force 
is  responsible  for  ASA  programming  and  budgeting  and  submits  the  ASA  request  as  part 
of  the  annual  Air  Force  budget  request.  OSD  will  continue  to  work  with  the  Secretary 
of  the  Air  Force  to  clarify  and  standardize  tracking  and  ASA  expenditures. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
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The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
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The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 

http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
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Connect  with  GAO 
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Relations 
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